Unmanned aerial vehicle image orthorectification method and device facing water surface

Abstract

The application provides a kind of unmanned aerial vehicle image ortho rectification method and device for water surface, applied to unmanned aerial vehicle surveying and mapping technical field, above-mentioned method includes: obtaining unmanned aerial vehicle image that unmanned aerial vehicle is shot to water surface area;Based on the longitude and latitude of photographic center, coordinate system conversion is carried out, and the plane coordinates of photographic center in photogrammetry coordinate system are obtained;According to collinear equation, based on the plane coordinates of photographic center and unified water surface elevation value, the ground point coordinates of each pixel in unmanned aerial vehicle image in photogrammetry coordinate system are determined;And based on the ground point coordinates of each pixel, the point cloud data of unmanned aerial vehicle image is constructed;Based on the geodetic height of unmanned aerial vehicle image, unified water surface elevation value and equivalent focal length when unmanned aerial vehicle is shot, ground sampling distance is determined;Based on ground sampling distance, the point cloud data of unmanned aerial vehicle image is ortho rasterization processing, and the image after ortho rectification is obtained.Through the present application, the defect that over-reliance on ground feature points in related art is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of unmanned aerial vehicle surveying and mapping technology, and in particular to a water surface-oriented unmanned aerial vehicle image orthorectification method and device. BACKGROUND

[0002] In the unmanned aerial vehicle aerial photography operation in the near-shore sea area, lake and other water surface areas, there are severe challenges due to the scene characteristics. Most of the aerial photography images in this area lack obvious ground feature characteristics, although there are ground objects such as aquaculture facilities in some areas, but due to the sparse distribution and similar feature height, the traditional feature point extraction and matching technology is difficult to accurately obtain a sufficient number of reliable feature points. If ground control points are used for auxiliary correction, a large amount of manpower and material resources are needed for the layout of the control points, and in the complex water surface environment, the measurement and marking of the control points are extremely difficult.

[0003] However, the existing unmanned aerial vehicle image orthorectification technology mostly relies on the extraction and matching of feature points, and the geometric relationship between the image and the real scene is established through these feature points, and then the orthorectification of the image is realized.

[0004] Due to the difficulty of meeting this technical requirement in the near-shore sea area, a large number of unmanned aerial vehicle aerial photography images cannot complete complete orthorectification, which makes it difficult to apply these images to actual marine surveying and mapping, environmental monitoring and other fields, resulting in waste of data resources. SUMMARY

[0005] The present application provides a water surface-oriented unmanned aerial vehicle image orthorectification method and device to solve the defects of the existing water surface-oriented unmanned aerial vehicle image orthorectification method, which relies too much on ground feature points, and improves the spatial consistency of the unmanned aerial vehicle image in the water surface area.

[0006] The present application provides a water surface-oriented unmanned aerial vehicle image orthorectification method, which comprises the following steps.

[0007] Obtaining an unmanned aerial vehicle image obtained by an unmanned aerial vehicle shooting a water surface area according to a preset flight route; performing coordinate system conversion based on the longitude and latitude of the photographic center of the unmanned aerial vehicle image to obtain the plane coordinates of the photographic center in the photogrammetric coordinate system; determining a unified water surface elevation value of the water surface area; determining the ground point coordinates of each pixel in the unmanned aerial vehicle image in the photogrammetric coordinate system based on the plane coordinates of the photographic center and the unified water surface elevation value according to the collinear equation; and constructing point cloud data of the unmanned aerial vehicle image based on the ground point coordinates of each pixel; determining the ground sampling distance based on the geodetic height of the unmanned aerial vehicle image, the unified water surface elevation value and the equivalent focal length when the unmanned aerial vehicle is shooting; and performing ortho-rasterization processing on the point cloud data of the unmanned aerial vehicle image based on the ground sampling distance to obtain the orthorectified image.

[0008] The method for orthorectifying water surface-oriented UAV images provided by the application comprises the following steps: determining a unified water surface elevation value of the water surface region based on a local horizontal plane of the water surface region and a tidal height.

[0009] The method for orthorectifying water surface-oriented UAV images provided by the application comprises the following steps: determining a unified water surface elevation value of the water surface region based on a local horizontal plane of the water surface region and a tidal height.

[0010] The method for orthorectifying water surface-oriented UAV images provided by the application comprises the following steps: determining a unified water surface elevation value of the water surface region based on a local horizontal plane of the water surface region and a tidal height.

[0011] ;

[0012] Wherein, represents the ground sampling distance, represents the geodetic height of the UAV image, represents the unified water surface elevation value, represents the equivalent focal length when the UAV is shooting.

[0013] The method for orthorectifying water surface-oriented UAV images provided by the application comprises the following steps: determining a unified water surface elevation value of the water surface region based on a local horizontal plane of the water surface region and a tidal height. :

[0014] ;

[0015] ;

[0016] Wherein, represents the horizontal coordinate of the current pixel in the photogrammetric coordinate system, represents the vertical coordinate of the current pixel in the photogrammetric coordinate system, represents the geodetic height of the ground point coordinate, represents the geodetic height of the UAV image, and respectively represent the horizontal coordinate and the vertical coordinate of the current pixel in the UAV image, and respectively represent the horizontal coordinate of the optical center of the UAV image and the vertical coordinate of the optical center of the UAV image, and respectively represent the horizontal coordinate and the vertical coordinate of the photographic center in the photogrammetric coordinate system; coefficients , 、 、 、 、 、 、 、 is a direction cosine element of a rotation matrix for describing a rotation relationship of an image space coordinate system of the UAV image relative to the photogrammetry coordinate system.

[0017] According to the present application, a water surface-oriented UAV image ortho-rectification method is provided, which is based on the ground sampling distance to perform ortho-rasterization processing on point cloud data of the UAV image to obtain an ortho-rectified image, including: determining the boundary of a regular grid based on the extreme values of the horizontal coordinates and the vertical coordinates of all ground point coordinates in the point cloud data of the UAV image; determining the starting coordinates and the number of grids of the regular grid based on the ground sampling distance and the boundary of the regular grid; constructing the regular grid based on the ground sampling distance, the starting coordinates and the number of grids; using a KD-tree-based nearest neighbor search and weighted interpolation algorithm to interpolate the point cloud data of the UAV image based on the regular grid to obtain an image attribute regular grid; and traversing the image attribute regular grid to solve each element in the image attribute regular grid to obtain the ortho-rectified image.

[0018] According to the water surface-oriented UAV image ortho-rectification method provided by the present application, for any grid point in the regular grid, the interpolation weight of the grid point is inversely proportional to the exponential power of the distance to the adjacent point.

[0019] According to the water surface-oriented UAV image ortho-rectification method provided by the present application, the construction of the regular grid based on the ground sampling distance, the starting coordinates and the number of grids includes: defining a regular grid , wherein each element corresponds to a grid coordinate , satisfying:

[0020] ;

[0021] ;

[0022] ;

[0023] wherein, denotes the horizontal coordinate of the starting coordinates, denotes the horizontal coordinate index of the grid coordinates in the regular grid, denotes the ground sampling distance, a longitudinal coordinate representing the starting coordinate, a longitudinal coordinate index representing a grid coordinate in the regular grid, a number of grids in a horizontal direction, a number of grids in a vertical direction.

[0024] The application further provides a water surface-oriented unmanned aerial vehicle image orthographic correction device, comprising the following modules: an acquisition module, configured to acquire unmanned aerial vehicle images obtained by an unmanned aerial vehicle shooting a water surface area according to a preset flight route; a conversion module, configured to perform coordinate system conversion based on longitude and latitude of a photography center of the unmanned aerial vehicle images, to obtain plane coordinates of the photography center in a photogrammetry coordinate system; a determination module, configured to determine a unified water surface elevation value of the water surface area; a point cloud module, configured to determine ground point coordinates of each pixel in the unmanned aerial vehicle images in the photogrammetry coordinate system according to a collinear equation, based on the plane coordinates of the photography center and the unified water surface elevation value, and to construct point cloud data of the unmanned aerial vehicle images based on the ground point coordinates of each pixel; a distance module, configured to determine a ground sampling distance based on a geodetic height of the unmanned aerial vehicle images, the unified water surface elevation value and an equivalent focal length when the unmanned aerial vehicle is shooting; and a correction module, configured to perform orthographic rasterization processing on the point cloud data of the unmanned aerial vehicle images based on the ground sampling distance, to obtain orthographically corrected images.

[0025] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the water surface-oriented unmanned aerial vehicle image orthographic correction method as described above when executing the computer program.

[0026] The application further provides a non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program is executable on a processor to implement the water surface-oriented unmanned aerial vehicle image orthographic correction method as described above.

[0027] The application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the water surface-oriented unmanned aerial vehicle image orthographic correction method as described above.

[0028] The unmanned aerial vehicle image orthorectification method and device for water surface provided by the application ensure the unity of the space reference and the accuracy of the geometric positioning by acquiring the unmanned aerial vehicle image and flight parameters and performing coordinate system conversion based on the longitude and latitude of the photographic center; the elevation modeling process of the approximate plane such as the water surface is effectively simplified by setting a unified water surface elevation value, and the calculation error caused by complex terrain undulations is avoided; the ground point coordinates are calculated pixel by pixel and the point cloud is constructed by combining the collinear equation and the unified elevation, so that the fine expression of the image geometric information is ensured; the resolution adaptive control is realized by introducing the geodetic height, water surface elevation and equivalent focal length to determine the ground sampling distance; and finally, the orthorectification is completed based on the sampling distance, so that the spatial consistency of the unmanned aerial vehicle image of the water surface area is significantly improved, and the orthorectification method has good practicability and engineering applicability. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0030] Figure 1 It is a flowchart of the unmanned aerial vehicle image orthorectification method for water surface provided by the application.

[0031] Figure 2 It is a module schematic diagram of the unmanned aerial vehicle image orthorectification device for water surface provided by the application.

[0032] Figure 3 It is a physical structure schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the application more clear, the technical scheme in the application will be clearly and completely described below in combination with the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0034] The application provides a feature point independent unmanned aerial vehicle image orthorectification method for water surface. The method is innovative and breaks through the limitation of traditional correction technology which depends on feature points. The method realizes the orthorectification of unmanned aerial vehicle image for water surface by fusing unmanned aerial vehicle flight parameters, water surface optical characteristics and geometric modeling technology, and provides key technical support for marine monitoring and water area surveying.

[0035] Optionally, the water surface oriented UAV image orthorectification method of the embodiment of the present application can be executed by a server, or by a terminal device, or by both the server and the terminal device. Taking the water surface oriented UAV image orthorectification method executed by the server as an example.

[0036] Figure 1 is a flowchart of the water surface oriented UAV image orthorectification method provided by the present application, as shown in the figure, the method comprises the following steps. Figure 1

[0037] Step 101, acquiring a UAV image obtained by a UAV shooting a water surface area according to a preset flight route.

[0038] In the embodiment of the present application, the UAV is controlled to take aerial photographs of the water surface area according to the preset flight route, and a JPG format UAV image with EXIF metadata is acquired, and flight trajectory and attitude data are recorded synchronously as flight parameters (including GPS coordinates, flight height, roll angle, pitch angle, yaw angle, etc.).

[0039] The EXIF data of the UAV image is analyzed by an ExifTool tool to acquire image scale information (width Width (unit: pixel), height Height (unit: pixel)), camera intrinsic parameters (equivalent focal length f (unit: pixel), optical center coordinates (x0, y0) (unit: pixel)), position information of the photographic center S (longitude lon_s (unit: degree), latitude lat_s (unit: degree), geodetic height h_s (unit: meter)), and attitude angles (roll angle (unit: degree), pitch angle (unit: degree), yaw angle (unit: degree)), etc.

[0040] In some embodiments, the preset flight route of the UAV is planned by a ground control station to ensure that it completely covers the target water surface area; the UAV carries an aerial camera and automatically flies according to the preset flight route, and takes vertical or oblique photographs of the water area at a set time interval or distance interval during the flight, and records high-precision POS data (including longitude and latitude, altitude) of each image, IMU attitude data (pitch, roll, heading angle), flight height, speed and GPS time stamp at the camera exposure moment synchronously; the UAV lands after completing the task, and the complete UAV image and the matching flight parameters are acquired through data export.

[0041] Step 102, performing coordinate system conversion based on the longitude and latitude of the photographic center of the UAV image to obtain the planar coordinates of the photographic center in the photogrammetric coordinate system. ​​​​​​

[0042] In the embodiment of the present application, the longitude (lon_s) and latitude (lat_s) of the photographic center of the unmanned aerial vehicle image are converted into plane coordinates (unit: meter) in the CGCS2000 Gauss-Kruger plane coordinate system through projection conversion.

[0043] Central meridian of the unmanned aerial vehicle image According to the image longitude calculation (3-degree zone projection):

[0044]

[0045] wherein, the central meridian of the unmanned aerial vehicle image is represented by, the lower limit rounding is represented by, the longitude of the photographic center of the unmanned aerial vehicle image is represented by.

[0046] Step 103, determining a unified water surface elevation value of the water surface region.

[0047] According to the unmanned aerial vehicle image orthorectification method for water surface provided by the present application, the determination of the unified water surface elevation value of the water surface region comprises:

[0048] Based on the local horizontal plane and the tidal height of the water surface region, the unified water surface elevation value of the water surface region is determined.

[0049] In the embodiment of the present application, when the unmanned aerial vehicle aerial photography region is located in the water surface region, there is no obvious terrain fluctuation, the water surface region is assumed to be a local horizontal plane, and based on the adjacent land elevation or water level station data, a unified water surface elevation value (such as the sum of the average horizontal plane and the tidal height) is set, and the influence of small waves is ignored.

[0050] Based on the monitoring data or geographic reference information of the water surface region, the real-time water level data measured by the adjacent land elevation point or recorded by the water level station is used as the basis, and the tidal change characteristics of the target region (if there is periodic tide) are combined to calculate the water surface average elevation reference value; finally, the water surface of the region is set as a unified elevation plane.

[0051] wherein, the tidal correction amount can be dynamically obtained through the tidal station data, and the static water level elevation is directly used in the non-tidal area. When implementing, the small elevation fluctuation (usually ≤0.1 meter) caused by waves is ignored to ensure the geometric stability of the water surface as a local horizontal plane.

[0052] Step 104, according to the collinear equation, based on the plane coordinates of the photographic center and the unified water surface elevation value, the ground point coordinates of each pixel in the unmanned aerial vehicle image in the photogrammetry coordinate system are determined; and based on the ground point coordinates of each pixel, the point cloud data of the unmanned aerial vehicle image is constructed.

[0053] ​In this embodiment of the invention, for any grid on the UAV image, its planar coordinates in the UAV image are (x, y), and the image pixel value is The coordinates of the corresponding ground point P in the photogrammetric coordinate system (a coordinate system used to describe the spatial position of ground objects; here, the CGCS2000 3-degree zone projected coordinate system is used) are: (Unit: meters).

[0054] According to the present invention, an orthorectification method for UAV images facing a water surface is provided. Based on the collinearity equation and the planar coordinates of the photogrammetric center and a uniform water surface elevation value, the method determines the ground point coordinates of each pixel in the UAV image in the photogrammetric coordinate system, including:

[0055] Take each pixel in the UAV image as the current pixel, and determine the ground point coordinates of the current pixel according to the following formula. :

[0056] ;

[0057] ;

[0058] in, This represents the x-coordinate of the current pixel in the photogrammetric coordinate system. This represents the ordinate of the current pixel in the photogrammetric coordinate system. The geodetic height represents the coordinates of a point on the ground. Indicates the elevation of the drone image. and These represent the x and y coordinates of the current pixel in the drone image, respectively. and These represent the x-coordinate and y-coordinate of the optical center of the UAV image, respectively. and These represent the x and y coordinates of the photogrammetric center in the photogrammetric coordinate system; coefficients , , , , , , , , It is a rotation matrix The direction cosine element is used to describe the rotation relationship between the image space coordinate system and the photogrammetric coordinate system of UAV imagery.

[0059] In this embodiment of the invention, the coordinates of the photography center in the photogrammetric coordinate system are ( (This refers to the spatial position of the center of the camera lens when this photo was taken.)

[0060] 、 、 (Where i = 1, 2, 3) is not an independent parameter, they together constitute a 3 rotation matrix R. This matrix is calculated from the three attitude angles of the camera (heading angle, pitch angle and roll angle).

[0061] 、 、 is the first column of the rotation matrix , representing the direction cosine of the X axis of the photogrammetry coordinate system under the camera coordinate system.

[0062] 、 、 is the second column of the rotation matrix , representing the direction cosine of the Y axis of the photogrammetry coordinate system under the camera coordinate system.

[0063] 、 、 is the third column of the rotation matrix , representing the direction cosine of the Z axis of the photogrammetry coordinate system under the camera coordinate system (usually pointing to the center of the earth).

[0064] represents the component of the vector from the projection center to the object point in the direction of the camera optical axis (Zc axis).

[0065] In the embodiment of the present application, given a pixel coordinate P(x, y) of a point on a UAV image, in the case of knowing the camera intrinsic parameters , the position and attitude of the camera at that time, and the approximate elevation of the point, the above formula can accurately calculate the ground point coordinate corresponding to the point (ground point P).

[0066] Using the above formula, a set of point cloud data can be obtained by traversing the image and repeating the above steps. , is the spatial coordinate, is the corresponding image pixel value.

[0067] Through the embodiment of the present application, the accurate mapping relationship between the unmanned aerial vehicle image pixel coordinates and the ground point coordinates of the photogrammetry coordinate system is constructed through a collinear equation, the plane coordinates of the photographic center, the unified water surface elevation value and the image posture parameters (direction cosine elements of the rotation matrix) are utilized, the geometric solution of the pixel level ground point coordinates is realized, the geometric deformation caused by the unmanned aerial vehicle posture change and the lens distortion is effectively eliminated, and the plane precision and the geometric fidelity of the water surface area orthographic image are improved.

[0068] In step 105, the ground sampling distance is determined based on the geodetic height of the unmanned aerial vehicle image, the unified water surface elevation value and the equivalent focal length when the unmanned aerial vehicle is photographed.

[0069] According to the unmanned aerial vehicle image orthographic correction method for a water surface provided by the present application, the ground sampling distance is determined based on the geodetic height of the unmanned aerial vehicle image, the unified water surface elevation value and the equivalent focal length when the unmanned aerial vehicle is photographed, and the method comprises the following steps:

[0070] The ground sampling distance is determined according to the following formula:

[0071] ;

[0072] Wherein, the ground sampling distance is represented by d, the geodetic height of the unmanned aerial vehicle image is represented by h, the unified water surface elevation value is represented by Z, and the equivalent focal length when the unmanned aerial vehicle is photographed is represented by f.

[0073] In the embodiment of the present application, three core parameters are first extracted from the unmanned aerial vehicle aerial photography system:

[0074] The photographic center geodetic height (i.e. the geodetic height of the unmanned aerial vehicle image, ): The real-time elevation data recorded by the GNSS / IMU system carried by the unmanned aerial vehicle is obtained, and the value represents the vertical distance (unit: meter) of the camera lens center relative to the reference ellipsoid at the photographic moment.

[0075] The unified water surface elevation value (Z): The determined static water surface elevation reference surface (unit: meter) is adopted, and the value is obtained through the water level station data or adjacent land elevation.

[0076] The equivalent focal length (f): The camera internal parameter (unit: pixel) parsed from the image EXIF metadata needs to be consistent with the scale calculation unit.

[0077] In the above formula, the numerator represents the vertical distance (i.e. the projection distance) from the photographic center to the water surface plane, and the denominator represents the equivalent focal length, which is used to determine the field of view angle of the image. The greater the focal length, the smaller the ground range corresponding to the unit pixel.

[0078] Through the embodiments of the present invention, a ground sampling distance (GSD) calculation model based on UAV imagery, uniform water surface elevation, and equivalent focal length is used to achieve a quantitative assessment of the spatial resolution of water surface area images. The formula intuitively reflects the linear influence of flight altitude difference and focal length on GSD, providing key spatial scale parameters for water surface orthorectification.

[0079] Step 106: Based on the ground sampling distance, perform orthorectification processing on the point cloud data of the UAV image to obtain the orthorectified image.

[0080] In this embodiment of the invention, the planar coordinates of all ground points in the point cloud data are first traversed, and the maximum and minimum values ​​in the X direction and the Y direction are extracted. These four extreme values ​​together define the spatial coverage of the point cloud.

[0081] The spatial extent is divided into a regular grid based on the ground sampling distance (GSD):

[0082] Horizontal direction (X-axis): Generates a mesh sequence with a step size GSD, starting coordinates The number of grids M is determined by dividing the range width by GSD and then rounding it up, using a floor-down alignment method.

[0083] Vertical direction (Y-axis): Similarly, generate a mesh sequence, starting coordinates. The number of grid cells L is calculated using the same logic.

[0084] Create a two-dimensional matrix (regular grid matrix) with M rows and L columns, where each grid cell (j,k) corresponds to a planar coordinate.

[0085] For the grid coordinates of each target grid point in the regular grid The KD-tree spatial indexing algorithm quickly searches for the n nearest neighbors (typically n=4~9) in the original point cloud data. The KD-tree organizes the point cloud into layers according to its spatial structure, which greatly improves the search efficiency.

[0086] Dynamically assign weights based on the spatial distance between neighboring points and the target grid point: Calculate the Euclidean distance from each neighboring point to the target grid point. Weight With distance of The power is inversely proportional to the power. In other words, the closer the neighboring points are, the greater their influence on the target grid point. The pixel value of the target grid point is obtained by weighted averaging of the pixel values ​​of its neighboring points, ensuring a smooth color transition.

[0087] By using regular grids and resampling operations, the geometric distortion caused by the drone's attitude tilt in the original image is completely eliminated, generating a corrected regular grid matrix.

[0088] The corrected rule grid matrix is bound with the CGCS2000 3-degree zone projection coordinate system and saved as a GeoTIFF format file.

[0089] The water surface-oriented unmanned aerial vehicle image ortho-correction method provided by the application is based on a ground sampling distance, and performs ortho-rasterization processing on point cloud data of an unmanned aerial vehicle image to obtain an ortho-corrected image, and comprises the following steps:

[0090] Determine the boundary of the regular grid based on the extreme values of the abscissa and the ordinate of all ground point coordinates in the point cloud data of the unmanned aerial vehicle image.

[0091] Determine the starting coordinates and the number of grids of the regular grid based on the ground sampling distance and the boundary of the regular grid.

[0092] Construct the regular grid based on the ground sampling distance, the starting coordinates and the number of grids.

[0093] Interpolate the point cloud data of the unmanned aerial vehicle image based on the regular grid by using a KD tree-based nearest neighbor search and weighted interpolation algorithm to obtain an image attribute regular grid.

[0094] Iterate through the image attribute regular grid, solve each element in the image attribute regular grid, and obtain the ortho-corrected image.

[0095] In the embodiment of the application, the boundary of the regular grid is determined as follows:

[0096] 、 、 、 ;

[0097] Wherein, represents the maximum value of the abscissa of the regular grid, represents the minimum value of the abscissa of the regular grid, represents the maximum value of the ordinate of the regular grid, represents the minimum value of the ordinate of the regular grid, represents the abscissa of the i-th ground point coordinate in the point cloud data of the unmanned aerial vehicle image, represents the ordinate of the i-th ground point coordinate in the point cloud data of the unmanned aerial vehicle image. represents the total number of ground points in the point cloud data.

[0098] Construct a regular grid sequence in the horizontal direction (X axis) :

[0099] ;

[0100] where the starting coordinate , the number of grids in the horizontal direction ;

[0101] Construct a regular grid sequence in the vertical direction (Y axis) :

[0102] ;

[0103] where the starting coordinate , the number of grids in the vertical direction . Indicates the lower limit rounding.

[0104] Through the embodiment of the present application, the spatial regularization processing of water surface point cloud data is realized by constructing a regular grid through ground sampling distance (GSD), the grid boundary is automatically determined by using extreme coordinates, the grid resolution is accurately controlled by combining GSD, and it is ensured that the corrected image and the actual water area scale are strictly matched; the grid hole problem caused by sparse distribution of point clouds is effectively solved by using KD tree accelerated neighbor search and weighted interpolation algorithm, and the continuity of water surface terrain expression is significantly improved.

[0105] According to the unmanned aerial vehicle image orthographic correction method for the water surface provided by the present application, a regular grid is constructed based on ground sampling distance, starting coordinates and the number of grids, including:

[0106] Define a regular grid , where each element corresponds to a grid coordinate , and satisfies:

[0107] ;

[0108] ;

[0109] ;

[0110] wherein, indicates the horizontal coordinate of the starting coordinate, indicates the horizontal coordinate index of the grid coordinate in the regular grid, indicates the ground sampling distance, indicates the vertical coordinate of the starting coordinate, indicates the vertical coordinate index of the grid coordinate in the regular grid, indicates the number of grids in the horizontal direction, indicates the number of grids in the vertical direction.

[0111] In the embodiment of the present application, the horizontal starting coordinate and the vertical starting coordinate The GSD integer times down is obtained by the point cloud boundary extreme value, respectively.

[0112] The horizontal direction (X axis) takes the horizontal starting coordinate as the starting point, and generates M coordinate values by increasing the step GSD;

[0113] The vertical direction (Y axis) takes the vertical starting coordinate as the starting point, and generates L coordinate values by increasing the step GSD;

[0114] A two-dimensional matrix (i.e. regular grid ) with LxM dimensions is constructed, wherein each element Corresponds to the grid coordinate , forming an equidistant regular grid covering the whole point cloud.

[0115] Through the embodiment of the application, the starting coordinates (horizontal starting coordinate , ) and the ground sampling distance (GSD) are used to realize the accurate spatial positioning of the grid, and the linear combination of the row index (j) and the column index (k) and the number of grids (M, L) are used to automatically generate the regular grid system covering the target water area, which not only ensures the consistency of the grid spacing and the image spatial resolution (GSD), but also eliminates the cumulative error through explicit coordinate calculation.

[0116] According to the unmanned aerial vehicle image orthographic correction method for the water surface provided by the application, for any grid point in the regular grid, the interpolation weight of the arbitrary grid point is inversely proportional to the weight exponential power of the distance to the adjacent point.

[0117] In the embodiment of the application, data resampling: based on the constructed regular grid, the original irregularly distributed point cloud data is interpolated to the grid.

[0118] For any grid , the n nearest points are searched by KD tree, wherein m is the nearest point index.

[0119] The weighted interpolation is used to calculate the grid value , wherein w is the weight, is the distance, is the weight index (usually ), is the original pixel value corresponding to the mth ground point in the point cloud. The regular grid is traversed, and the solution of each element is completed, that is, the orthographic correction of the image is completed.

[0120] Geocoding storage: the CGCS2000 3 degree band projection coordinate system is adopted, and the central meridian is​ The ortho-corrected image is saved as a GeoTIFF format file.

[0121] By adopting the interpolation weight distribution strategy of the weight index power being inversely proportional to the distance of adjacent points, the spatial adaptive weighted interpolation of the regular grid points is realized, so that the point cloud data of the points closer to the grid point to be interpolated contributes more to the correction result, and the interference of the distant noise points is effectively inhibited.

[0122] The water surface-oriented unmanned aerial vehicle image ortho-correction device provided by the present application is described below, and the water surface-oriented unmanned aerial vehicle image ortho-correction device described below can be correspondingly referred to the water surface-oriented unmanned aerial vehicle image ortho-correction method described above.

[0123] Reference Figure 2 , Figure 2 is a module schematic diagram of the water surface-oriented unmanned aerial vehicle image ortho-correction device provided by the present application.

[0124] The acquisition module 201 is configured to acquire the unmanned aerial vehicle image obtained by the unmanned aerial vehicle shooting the water surface area according to a preset flight route;

[0125] The conversion module 202 is configured to perform coordinate system conversion based on the longitude and latitude of the photographic center of the unmanned aerial vehicle image to obtain the plane coordinates of the photographic center in the photogrammetry coordinate system;

[0126] The determination module 203 is configured to determine a unified water surface elevation value of the water surface area;

[0127] The point cloud module 204 is configured to determine the ground point coordinates of each pixel in the unmanned aerial vehicle image in the photogrammetry coordinate system based on the plane coordinates of the photographic center and the unified water surface elevation value according to the collinear equation, and construct the point cloud data of the unmanned aerial vehicle image based on the ground point coordinates of each pixel;

[0128] The distance module 205 is configured to determine the ground sampling distance based on the geodetic height of the unmanned aerial vehicle image, the unified water surface elevation value, and the equivalent focal length when the unmanned aerial vehicle is shooting.

[0129] The correction module 206 is configured to perform ortho-rasterization processing on the point cloud data of the unmanned aerial vehicle image based on the ground sampling distance to obtain the ortho-corrected image.

[0130] Specifically, the above water surface-oriented unmanned aerial vehicle image ortho-correction device provided by the present application can realize all method steps realized by the above water surface-oriented unmanned aerial vehicle image ortho-correction method embodiment, and can achieve the same technical effects. The same parts and beneficial effects in this embodiment as the method embodiment will not be described in detail.

[0131] Figure 3This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communications bus 340. The processor 310 can call logic instructions in the memory 330 to execute a method for orthorectifying UAV images facing a water surface. This method includes: acquiring UAV images taken by the UAV along a preset flight path over a water surface area; performing coordinate system transformation based on the latitude and longitude of the UAV image's photographic center to obtain the planar coordinates of the photographic center in a photogrammetric coordinate system; determining a uniform water surface elevation value for the water surface area; determining the ground point coordinates of each pixel in the UAV image in the photogrammetric coordinate system based on the collinearity equation, the planar coordinates of the photographic center, and the uniform water surface elevation value; constructing point cloud data of the UAV image based on the ground point coordinates of each pixel; determining the ground sampling distance based on the UAV image's geodetic height, the uniform water surface elevation value, and the equivalent focal length during UAV shooting; and performing orthorasterization processing on the point cloud data of the UAV image based on the ground sampling distance to obtain the orthorectified image.

[0132] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer readable storage medium, and the computer program being capable of executing the water surface oriented unmanned aerial vehicle image orthorectification method provided by the above method when executed by a processor, the method comprising: obtaining unmanned aerial vehicle images obtained by an unmanned aerial vehicle shooting a water surface area according to a preset flight route; performing coordinate system conversion based on the longitude and latitude of a photographing center of the unmanned aerial vehicle images to obtain plane coordinates of the photographing center in a photogrammetry coordinate system; determining a unified water surface elevation value of the water surface area; determining ground point coordinates of each pixel in the unmanned aerial vehicle images in the photogrammetry coordinate system based on the plane coordinates of the photographing center and the unified water surface elevation value according to a collinearity equation; and constructing point cloud data of the unmanned aerial vehicle images based on the ground point coordinates of each pixel; determining a ground sampling distance based on the geodetic height of the unmanned aerial vehicle images, the unified water surface elevation value and an equivalent focal length when the unmanned aerial vehicle shoots; and performing ortho-rasterization processing on the point cloud data of the unmanned aerial vehicle images based on the ground sampling distance to obtain orthorectified images.

[0134] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, the computer program being capable of implementing the water surface oriented unmanned aerial vehicle image orthorectification method provided by the above method when executed by a processor, the method comprising: obtaining unmanned aerial vehicle images obtained by an unmanned aerial vehicle shooting a water surface area according to a preset flight route; performing coordinate system conversion based on the longitude and latitude of a photographing center of the unmanned aerial vehicle images to obtain plane coordinates of the photographing center in a photogrammetry coordinate system; determining a unified water surface elevation value of the water surface area; determining ground point coordinates of each pixel in the unmanned aerial vehicle images in the photogrammetry coordinate system based on the plane coordinates of the photographing center and the unified water surface elevation value according to a collinearity equation; and constructing point cloud data of the unmanned aerial vehicle images based on the ground point coordinates of each pixel; determining a ground sampling distance based on the geodetic height of the unmanned aerial vehicle images, the unified water surface elevation value and an equivalent focal length when the unmanned aerial vehicle shoots; and performing ortho-rasterization processing on the point cloud data of the unmanned aerial vehicle images based on the ground sampling distance to obtain orthorectified images.

[0135] The device embodiments described above are only schematic, wherein units shown as separate components can or can not be physically separate, and components shown as units can or can not be physical units, i.e., can be located in one place or distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement without creative labor.

[0136] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for orthorectifying images of a water surface facing unmanned aerial vehicle, characterized in that, The method comprises the following steps: acquiring a UAV image obtained by a UAV shooting a water surface area according to a preset flight route; performing coordinate system conversion based on the longitude and latitude of a photographing center of the UAV image to obtain a plane coordinate of the photographing center in a photogrammetry coordinate system; determining a unified water surface elevation value of the water surface area; determining, according to a collinearity equation, a ground point coordinate of each pixel in the UAV image in the photogrammetry coordinate system based on the plane coordinate of the photographing center and the unified water surface elevation value, and constructing point cloud data of the UAV image based on the ground point coordinate of each pixel; determining a ground sampling distance based on the height of the UAV image, the unified water surface elevation value and an equivalent focal length when the UAV is shooting; performing ortho-rasterization processing on the point cloud data of the UAV image based on the ground sampling distance to obtain an ortho-corrected image.

2. The water-facing drone image orthorectification method of claim 1, wherein, The method for determining the unified water surface elevation value of the water surface area comprises the following steps: determining the unified water surface elevation value of the water surface area based on a local horizontal plane and a tidal height of the water surface area; The method for determining the ground sampling distance based on the height of the UAV image, the unified water surface elevation value and the equivalent focal length when the UAV is shooting comprises the following steps: ； wherein, denotes the ground sampling distance, denotes the geodetic height of the drone image, denotes the unified water surface elevation value, denotes the equivalent focal length at the time of drone shooting. 3.The water-facing UAV image orthorectification method of claim 1, wherein, The method for determining, according to a collinearity equation, a ground point coordinate of each pixel in the UAV image in the photogrammetry coordinate system based on the plane coordinate of the photographing center and the unified water surface elevation value comprises the following steps: determining ground point coordinates of each pixel in the unmanned aerial vehicle image according to the following formula : ； ； wherein, denotes the horizontal coordinate of the current pixel in the photogrammetric coordinate system, denotes the vertical coordinate of the current pixel in the photogrammetric coordinate system, denotes the geodetic height of the ground point coordinate, denotes the geodetic height of the UAV image, and denote the horizontal and vertical coordinates of the current pixel in the UAV image, respectively, and denote the horizontal and vertical coordinates of the optical center of the UAV image, respectively, and denote the horizontal and vertical coordinates of the photographic center in the photogrammetric coordinate system, respectively; coefficients , , , , , , , , are direction cosine elements of a rotation matrix for describing the rotation relationship of the image space coordinate system of the UAV image relative to the photogrammetric coordinate system. 4.The water-facing UAV image orthorectification method of claim 1, wherein, The method for performing ortho-rasterization processing on the point cloud data of the UAV image based on the ground sampling distance to obtain an ortho-corrected image comprises the following steps: determining a boundary of a regular grid based on the extreme values of the abscissa and the ordinate of all ground point coordinates in the point cloud data of the UAV image; determining a starting coordinate and a grid number of the regular grid based on the ground sampling distance and the boundary of the regular grid; constructing the regular grid based on the ground sampling distance, the starting coordinate and the grid number; performing interpolation on the point cloud data of the UAV image based on the regular grid by using a KD-tree-based nearest neighbor search and weighted interpolation algorithm to obtain an image attribute regular grid; iterating through the image attribute regular grid to solve each element in the image attribute regular grid to obtain an ortho-corrected image.

5. The water-facing drone image orthorectification method of claim 4, wherein, For any grid point in the regular grid, the interpolation weight of the any grid point is inversely proportional to the exponential power of the distance to the adjacent point.

6. The water-facing UAV image orthorectification method of claim 4, wherein, The method for constructing the regular grid based on the ground sampling distance, the starting coordinate and the grid number comprises the following steps: defining a grid of rules where each element corresponding grid coordinates satisfying: ； ； ； wherein denotes the horizontal coordinate of the start coordinate, denotes the horizontal coordinate index of the grid coordinate in the regular grid, denotes the ground sampling distance, denotes the vertical coordinate of the start coordinate, denotes the vertical coordinate index of the grid coordinate in the regular grid, denotes the number of grids in horizontal direction, denotes the number of grids in vertical direction.

7. An apparatus for orthorectifying images of a water surface-oriented unmanned aerial vehicle, characterized by, The method comprises the following steps: an acquisition module configured to acquire a UAV image obtained by a UAV shooting a water surface area according to a preset flight route; a conversion module configured to perform coordinate system conversion based on the longitude and latitude of a photographing center of the UAV image to obtain a plane coordinate of the photographing center in a photogrammetry coordinate system; a determination module configured to determine a unified water surface elevation value of the water surface area; a point cloud module configured to determine ground point coordinates of each pixel in the UAV image in the photogrammetry coordinate system according to a collinearity equation based on the planar coordinates of the photographic center and the unified water surface elevation value, and construct point cloud data of the UAV image based on the ground point coordinates of each pixel; a distance module configured to determine a ground sampling distance based on the geodetic height of the UAV image, the unified water surface elevation value, and an equivalent focal length when the UAV is shooting; a correction module configured to ortho-rasterize the point cloud data of the UAV image based on the ground sampling distance to obtain ortho-corrected images.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the ortho-correction method for water surface-oriented UAV images according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the ortho-correction method for water surface-oriented UAV images according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the ortho-correction method for water surface-oriented UAV images according to any one of claims 1 to 6.

Images