Target area-oriented oblique photogrammetry image automatic screening method

Abstract

The invention discloses a target area-oriented oblique photogrammetry image automatic screening method, and belongs to the technical field of oblique photogrammetry and three-dimensional modeling. The technical problem to be solved is to solve the problem that aerial triangulation is easy to fail in weak texture areas such as a water area and a bare land in a measurement area. According to the technical scheme, the method is characterized in that a photogrammetry principle and a GIS spatial analysis technology are fused, a localized elevation datum plane is established through pre-flight calibration to calculate the relative flight height, an image geographical range area domain is generated through coordinate transformation, rotation matrix construction and the like, and then an optimal image subset associated with a target area is automatically screened out through triple spatial topological relation judgment, so that the target area can be accurately identified. And invalid image interference is avoided from the source.

Description

Technical Field

[0001] This invention belongs to the field of oblique photogrammetry and 3D modeling technology, and specifically relates to an automatic selection method for oblique photogrammetry images oriented towards a target area. Background Technology

[0002] Oblique photogrammetry, as a core tool in modern surveying and 3D modeling, has been widely applied in fields such as urban planning, geological disaster monitoring, cultural heritage protection, and engineering construction. It acquires high-resolution images from multiple perspectives using drones or aerial platforms equipped with multi-angle lenses, and then uses aerial triangulation to achieve precise image orientation and generate 3D point clouds, thereby constructing high-precision real-world 3D models. However, in practical applications, if the survey area contains areas with weak or repetitive textures such as water bodies, bare land, snowfields, or sandy areas, the lack of sufficient feature points can lead to difficulties in image matching, resulting in aerial triangulation failures or distorted results. Existing technologies, such as the "full-scale aerial triangulation iterative method," require repeated trial and error to eliminate invalid images, which is time-consuming and labor-intensive; the "manual experience pre-screening method" is inefficient and highly subjective, resulting in poor consistency and repeatability of the results.

[0003] Relevant patent documents retrieved:

[0004] Country of Origin: China, Publication Number: CN107917699B, Publication Date: January 17, 2020. This document discloses a method for improving the aerial triangulation quality of oblique photogrammetry in mountainous areas. Specifically, it includes: determining the absolute flight altitude (including several identical or different altitudes) for each flight path based on ground resolution requirements and the undulation of the mountainous area, and obtaining the relative and absolute flight altitudes for each flight path; acquiring oblique image data, GNSS data, and IMU data of the mountainous area through aerial remote sensing flight; processing the above data; and employing overall aerial triangulation (applying it to all data). The aerial triangulation results and reports are obtained by either performing aerial triangulation on the data in blocks and then merging the blocks (performing aerial triangulation on the data for each day separately, merging the merged data, and then performing aerial triangulation on the merged data). Both aerial triangulation methods include SURF feature point detection, SURF feature point description, and RANSAC exact matching. The block aerial triangulation is performed under the same time phase and the same tilt angle, while the overall aerial triangulation is performed under different time phases and different tilt angles. The aerial triangulation results and reports under different conditions are compared, and the optimal result is selected for 3D modeling of mountainous real-world scenes to form clear and accurate 3D geographic information data.

[0005] The prior art represented by the aforementioned documents has at least the following unresolved technical problems or defects: (1) No solution is provided for the failure of aerial triangulation in weak / repeating texture regions: This paper relies on SURF feature point detection and matching to achieve aerial triangulation. However, weak / repeating texture regions lack sufficient feature points, which still leads to difficulties in image matching and failure to generate effective connection points, ultimately causing the aerial triangulation calculation to be interrupted or the results to be distorted. This core technical problem has not been solved. (2) The automatic screening of images is not realized, and the computational resources are consumed and the efficiency is low: This paper requires all the acquired oblique image data to be subjected to overall aerial triangulation, or to process all the data in blocks and then merge the aerial triangulation. The invalid / interference images covering weak texture areas are not removed from the source of the data. There are still a lot of redundant data involved in the calculation, which makes the aerial triangulation process time-consuming and labor-intensive, and it is difficult to meet the timeliness requirements of large-scale engineering projects. (3) No spatial relationship intelligent discrimination mechanism, unable to accurately focus on the target area: This paper does not involve the spatial topological relationship discrimination between the image and the target area, and does not select the image subset that is only related to the target area. It still needs to process the image data of non-target areas in the survey area, which further increases the computational complexity. Moreover, it does not solve the subjective problem of relying on manual experience in the traditional method. (4) Lack of local adaptability in relative flight altitude calculation: This paper determines the relative flight altitude based on ground resolution requirements and mountain undulations, without establishing a localized elevation datum for specific flight missions. This makes it difficult to adapt to the terrain features of different survey areas, limiting the accuracy of the relative flight altitude calculation and potentially affecting the reliability of subsequent aerial triangulation results.

[0006] In view of this, the present invention is hereby proposed. Summary of the Invention

[0007] To address the aforementioned technical problems in the existing technology, this invention provides an automatic screening method for oblique photogrammetry images oriented towards a target area, solving the problem of aerial triangulation failure in areas such as water bodies and areas with weak textures.

[0008] To achieve the above objectives, the technical solution of the present invention is as follows: An automatic selection method for oblique photogrammetric images oriented towards a target area includes: S1. Relative flight altitude calculation: Establish a localized elevation datum and calculate the relative flight altitude of each aerial image relative to the localized elevation datum. S2. Image geographic range construction: Based on photogrammetry principles and coordinate transformation technology, the ground coverage geographic range area of ​​each aerial image is obtained by sequentially performing coordinate transformation, attitude parameter processing, rotation matrix construction, and corner point geodetic coordinate calculation. S3. Intelligent spatial relationship identification: After preprocessing the target area vector red line, the system uses multiple spatial topological relationship identification methods to filter out and output images that are spatially related to the target area.

[0009] Furthermore, step S1 specifically includes: Pre-flight calibration involves taking pre-flight photos in the survey area to acquire calibration images, extracting and calculating the absolute flight altitude values ​​from the POS data of each calibration image, and calculating the arithmetic mean of the absolute flight altitude values ​​as the localized elevation datum. Calculate the relative flight altitude by obtaining the absolute flight altitude value from the POS data of each aerial photograph taken in the official aerial photography, and then calculate the relative flight altitude H of that aerial photograph using the following formula:

[0010] in, This is the absolute flight altitude value. This is a localized elevation datum.

[0011] Furthermore, the coordinate transformation specifically includes: The WGS84 latitude and longitude coordinates recorded in the POS data are converted into plane coordinates in the National Geodetic 2000 coordinate system based on the WGS84 and CGCS2000 ellipsoid parameters.

[0012] Furthermore, the attitude parameter processing specifically includes: The attitude angles Omega, Phi, and Kappa are extracted from the POS data, and the units of the attitude angles are converted to radians.

[0013] Furthermore, the construction of the rotation matrix specifically includes: Based on the collinearity equation principle and the three-dimensional spatial rotation transformation theory of photogrammetry, the primitive rotation matrices around the Z-axis, Y-axis and X-axis are constructed in the order of Yaw-Pitch-Roll, and the platform attitude rotation matrix is ​​synthesized by matrix chain multiplication; combined with the inherent mounting angle of the UAV lens, the lens mounting rotation matrix is ​​constructed. Multiply the lens mounting rotation matrix by the platform attitude rotation matrix to obtain the total rotation matrix Rtotal from the image space coordinate system to the local horizontal coordinate system.

[0014] Furthermore, the calculation of the corner point geodetic coordinates in step S2 specifically includes: In the image space coordinate system, coordinate vectors of four corner points are defined for each lens of the UAV. The coordinate vector format is [x, y, -f], where x and y are normalized coordinates based on the half-sensor size, and f is the lens focal length. Using the total rotation matrix Rtotal, the coordinate vector of the corner point is rotated to the local horizontal coordinate system to obtain a direction vector V containing three components: east, north, and sky. Based on the idea of ​​collinearity equations, the intersection of the camera projection center, the direction vector V, and the ground plane is calculated to obtain the geodetic coordinates of each corner point.

[0015] Furthermore, in step S2, all the lenses of the UAV and the four corner points of each lens are traversed, and the geodetic coordinates of the corner points are connected in sequence to form the geographical area corresponding to each aerial image.

[0016] Furthermore, the target area vector red line preprocessing in step S3 specifically includes: The vector red line of the target area is converted into a set of point coordinates, and then the point coordinates are converted from WGS84 latitude and longitude coordinates to plane coordinates under the National Geodetic 2000 coordinate system.

[0017] Furthermore, in step S3, the determination of multiple spatial topological relationships is performed in the following order: S31. Point position determination: The ray method is used to detect whether at least one of the four corner points of the geographic area of ​​the aerial image falls inside the vector red line polygon. If so, it is determined that there is a spatial relationship. S32. Reverse containment detection: If S31 determines that there is no correlation, check whether any vertex of the vector red line polygon falls inside the geographic area of ​​the aerial image. If so, it is determined that there is a spatial correlation. S33. Polygon intersection detection: If S31 and S32 both determine that there is no correlation, check whether there is an intersection between the geographic area of ​​the aerial image and the boundary of the vector red line polygon. If so, it is determined that there is a spatial correlation.

[0018] Furthermore, before performing the multi-space topology relationship discrimination in step S3, the vector red line of the target area is expanded by a preset distance to generate a buffer polygon, which is used as the boundary for spatial topology relationship discrimination. Alternatively, in the corner point geodetic coordinate calculation in step S2, the ground plane is replaced with a digital elevation model (DEM), and the intersection of the direction vector V and the real ground surface represented by the DEM is calculated.

[0019] The beneficial effects of this invention are as follows: (1) Improve the success rate and stability of aerial triangulation: By selectively removing invalid and interfering images covering weak texture areas such as water areas and bare land, the adverse factors that cause aerial triangulation matching failures are greatly reduced from the data source, and the aerial triangulation of complex areas such as islands is transformed from frequent failures to stable success, which significantly improves the success rate and operational stability of aerial triangulation. (2) Improve data processing efficiency and reduce computing costs: By using fully automated image screening, the total number of images involved in aerial triangulation calculation is effectively reduced, which directly reduces the computational complexity of aerial triangulation solution and model reconstruction, significantly shortens the overall data processing cycle, and saves a lot of computing hardware resources and time costs, thereby improving data processing efficiency and reducing overall computing costs. (3) Improve the accuracy and completeness of image screening: The triple spatial relationship discrimination mechanism based on ray method point position discrimination, reverse inclusion detection and polygon intersection detection can accurately identify various complex spatial relationships such as intersection, inclusion and tangency between images and target areas, effectively avoid the problem of missing effective images and misselecting invalid images, provide a high-quality data foundation for aerial triangulation calculation, and significantly improve the accuracy and completeness of image screening. (4) Realize the automation and intelligence of the process flow and reduce the dependence on manual labor: The traditional manual image screening process that relies on the experience and judgment of operators is transformed into a fully automated processing process, which avoids human error and subjective judgment differences, ensures the repeatability and standardization of screening results, realizes the automation and intelligence of the process flow, and greatly reduces the dependence on highly skilled professionals. (5) It has strong versatility and scalability: This method has wide applicability and good scalability. It is not only applicable to the three-dimensional modeling of island scenes, but also widely used in various scenarios that require focused modeling of specific areas, such as building fine modeling, geological disaster monitoring, and archaeological site digitization. It provides a general technical framework for solving similar spatial target focusing image screening problems. Attached Figure Description

[0020] Figure 1 A flowchart of an automatic image selection method for oblique photogrammetry provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are not all embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0022] It should be noted that, unless otherwise specifically stated, the relative arrangement and numerical expressions of the components and steps described in these embodiments should not be construed as limiting the scope of the invention.

[0023] The following description of exemplary embodiments is merely illustrative and is not intended to limit the invention or its application or use in any way. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail herein, but where applicable, such techniques, methods, and apparatus should be considered part of this specification.

[0024] Example See Figure 1 , Figure 1This is a flowchart of an automatic image selection method for oblique photogrammetry targeting a specific area, proposed in this invention. It integrates GIS spatial analysis technology with photogrammetry principles, and intelligently selects the optimal image subset through automated image-region spatial relationship discrimination. Its core technical solution mainly includes three components: relative flight altitude calculation, image geographic extent construction, and intelligent spatial relationship discrimination. Specific steps may include: S1. Relative flight altitude calculation: Establish a localized elevation datum and calculate the relative flight altitude of each aerial image relative to the localized elevation datum; specifically including: S11. At the initial stage of the flight mission, pre-flight calibration is performed. Pre-flight photography is conducted in the survey area to acquire several (e.g., 4) calibration images. The absolute flight altitude values ​​in the POS data of each calibration image are extracted and calculated. The arithmetic mean of the absolute flight altitude values ​​is calculated and used as the localized elevation datum. ; S12. During the actual aerial photography process, for each aerial image, calculate the relative flight altitude and obtain the absolute flight altitude value from the POS data of each aerial image taken during the actual aerial photography. The relative flight altitude H of the aerial image is calculated using the following formula:

[0025] in, This is the absolute flight altitude value. This is a localized elevation datum.

[0026] S2. Image geographic extent construction: Based on photogrammetry principles and coordinate transformation technology, the process sequentially involves coordinate transformation, attitude parameter processing, rotation matrix construction, and corner geodetic coordinate calculation to obtain the ground coverage geographic extent of each aerial image; specifically including: S21. The coordinates of the photos recorded by POS are latitude and longitude coordinates under WGS84. First, according to the WGS84 and CGCS2000 ellipsoid parameters, the latitude and longitude coordinates are converted into plane coordinates under the National Geodetic 2000 coordinate system. Then, the plane coordinates, absolute flight altitude and attitude angles (Omega, Phi, Kappa) are extracted from the POS data, and the units of the attitude angles are converted to radians. S22. Based on the collinearity equation principle and the three-dimensional spatial rotation transformation theory in photogrammetry, according to... The primitive rotation matrices for rotation around the Z-axis, Y-axis, and X-axis are constructed in the order of (yaw-pitch-roll), and then synthesized into a complete three-dimensional attitude rotation matrix from the body coordinate system to the local horizontal coordinate system through matrix chain multiplication. ; Meanwhile, the inherent mounting angle rotation matrices of the five lenses of the drone have a fixed configuration, with the mounting rotation matrix of the downward-looking lens being an identity matrix, and the mounting rotation matrices of the forward-tilting lens and the rear-looking lens rotating around the X-axis respectively. and The matrices for the left-tilt and right-tilt lenses are respectively rotated around the Y-axis. and Matrix; S23. For each lens, match its corresponding mounting rotation matrix with the platform attitude rotation matrix. Multiplying these matrices yields the final transformation matrix from the image space coordinate system to the local horizontal coordinate system. The final transformation matrix The specific formula is:

[0027]

[0028]

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] in, , , These are the elementary rotation matrices about the Z, Y, and X axes, respectively. , , These are the angles around the Z, Y, and X axes, respectively. It is the rotation matrix of the camera lens relative to the fixed mounting angle of the camera body. For the rotation matrix of the front-view and rear-view cameras, For the rotation matrices of the left and right view cameras, all matrix operations use matrix multiplication instead of dot product.

[0035] S24. In the image space coordinate system, define the coordinate vectors of the four corner points for each lens of the drone (in the format of...). ,in and For normalized image plane coordinates based on half-sensor size, (This represents the camera focal length corresponding to the lens; the negative sign indicates the direction of the optical axis.)

[0036] For each corner point, the total rotation matrix through its corresponding lens... Performing matrix multiplication on its coordinate vector rotates it from the image space coordinate system to the local horizontal coordinate system, yielding a direction vector containing east, north, and sky components. Based on the concept of collinearity equations, and combined with the east coordinate of the camera projection center in the object coordinate system... North coordinates and relative flight altitude The camera center and the direction vector The intersection calculation is performed with the ground plane (takeoff point elevation) to obtain the final geodetic coordinates (east coordinates) of that corner point. North coordinates ).

[0037] After traversing all five lenses of the drone and the four corner points of each lens, and calculating the geodetic coordinates of all corner points, the four corner points corresponding to each lens are connected sequentially to form five irregular quadrilateral geographic regions corresponding to a single five-lens image. Each region is composed of four corner points with east and north coordinates, providing high-precision input data for subsequent spatial overlay analysis. The comprehensive calculation formula for calculating the geodetic coordinates of the corner points using the collinearity equation is as follows:

[0038]

[0039] in, and Let represent the east and north coordinates of the object-space coordinate system calculated from the collinearity equations, respectively. , and This represents the east and north coordinates of the camera projection center and the relative flight altitude in the object coordinate system. It is a vector that represents the precise position of a point in a spatial coordinate system. and Represents image plane coordinates, This indicates the focal length of the lens corresponding to the camera. Represents the celestial component of a vector. Represents the eastern component of a vector. The north component of a vector is represented; all matrix operations use matrix multiplication instead of dot product.

[0040] S3. Intelligent spatial relationship identification: After preprocessing the target area vector red line, it uses multiple spatial topological relationship identification methods to filter out and output images that are spatially related to the target area; specifically including: Since the vector red line is composed of points connected end to end, it is first transformed into a set of multiple point coordinates. Because the point coordinates of the vector red line are WGS84 latitude and longitude coordinates, following the same rules as image coordinate transformation, and based on the WGS84 and CGCS2000 ellipsoid parameters, these latitude and longitude coordinates are converted into planar coordinates in the National Geodetic 2000 coordinate system to ensure coordinate system consistency. After preprocessing, the following multiple spatial topological relationship determination steps are performed on each aerial image: S31. Using a point position detection algorithm based on ray method, determine whether at least one of the four corner points of the area covered by the image falls inside the vector red line polygon; if so, immediately determine that the image overlaps with the target area and proceed to the result statistics stage. S32. If S31 determines that there is no overlap, then perform reverse inclusion relationship detection to determine whether any vertex of the vector red line polygon is located inside the area covered by the image; if it is, then determine that the image overlaps with the target area and proceed to the result statistics stage. S33. If no overlap is detected in S31 and S32, then the polygon intersection detection algorithm is executed to calculate whether there are one or more intersection points between the image coverage area and the boundary of the vector red line polygon; if an intersection point is detected, then it is determined that the image overlaps with the target area, and the result statistics are entered. S34. Once a spatial relationship between an image and the target area is confirmed using any of the discrimination methods S31, S32, and S33, the image is marked as selected. After all images have been discriminated, a list containing the indices of all selected images is generated as the output. This triple discrimination mechanism ensures accurate identification even if the image coverage area only has boundary contact or partial inclusion with the target area, providing the optimal image data set for subsequent aerial triangulation calculations.

[0041] The above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to examples, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An automatic selection method for oblique photogrammetric images oriented towards a target area, characterized in that, include: S1. Relative flight altitude calculation: Establish a localized elevation datum and calculate the relative flight altitude of each aerial image relative to the localized elevation datum. S2. Image geographic range construction: Based on photogrammetry principles and coordinate transformation technology, the ground coverage geographic range area of ​​each aerial image is obtained by sequentially performing coordinate transformation, attitude parameter processing, rotation matrix construction, and corner point geodetic coordinate calculation. S3. Intelligent spatial relationship identification: After preprocessing the target area vector red line, the system uses multiple spatial topological relationship identification methods to filter out and output images that are spatially related to the target area.

2. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 1, characterized in that, Step S1 specifically includes: Pre-flight calibration involves taking pre-flight photos in the survey area to acquire calibration images, extracting and calculating the absolute flight altitude values ​​from the POS data of each calibration image, and calculating the arithmetic mean of the absolute flight altitude values ​​as the localized elevation datum. Calculate the relative flight altitude by obtaining the absolute flight altitude value from the POS data of each aerial photograph taken in the official aerial photography, and then calculate the relative flight altitude H of that aerial photograph using the following formula: in, This is the absolute flight altitude value. This is a localized elevation datum.

3. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 1, characterized in that, The coordinate transformation specifically includes: The WGS84 latitude and longitude coordinates recorded in the POS data are converted into plane coordinates in the National Geodetic 2000 coordinate system based on the WGS84 and CGCS2000 ellipsoid parameters.

4. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 3, characterized in that, The attitude parameter processing specifically includes: The attitude angles Omega, Phi, and Kappa are extracted from the POS data, and the units of the attitude angles are converted to radians.

5. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 1, characterized in that, The construction of the rotation matrix specifically includes: Based on the collinearity equation principle and the three-dimensional spatial rotation transformation theory of photogrammetry, the primitive rotation matrices around the Z-axis, Y-axis and X-axis are constructed in the order of Yaw-Pitch-Roll, and the platform attitude rotation matrix is ​​synthesized by matrix chain multiplication; combined with the inherent mounting angle of the UAV lens, the lens mounting rotation matrix is ​​constructed. Multiply the lens mounting rotation matrix by the platform attitude rotation matrix to obtain the total rotation matrix Rtotal from the image space coordinate system to the local horizontal coordinate system.

6. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 5, characterized in that, The calculation of the geodetic coordinates of the corner point in step S2 specifically includes: In the image space coordinate system, coordinate vectors of four corner points are defined for each lens of the UAV. The coordinate vector format is [x, y, -f], where x and y are normalized coordinates based on the half-sensor size, and f is the lens focal length. Using the total rotation matrix Rtotal, the coordinate vector of the corner point is rotated to the local horizontal coordinate system to obtain a direction vector V containing three components: east, north, and sky. Based on the idea of ​​collinearity equations, the intersection of the camera projection center, the direction vector V, and the ground plane is calculated to obtain the geodetic coordinates of each corner point.

7. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 6, characterized in that, In step S2, all the lenses of the UAV and the four corner points of each lens are traversed, and the geodetic coordinates of the corner points are connected in sequence to form the geographical area corresponding to each aerial image.

8. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 1, characterized in that, The target area vector redline preprocessing in step S3 specifically includes: The vector red line of the target area is converted into a set of point coordinates, and then the point coordinates are converted from WGS84 latitude and longitude coordinates to plane coordinates under the National Geodetic 2000 coordinate system.

9. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 8, characterized in that, In step S3, the determination of multiple spatial topological relationships is performed in the following order: S31. Point position determination: The ray method is used to detect whether at least one of the four corner points of the geographic area of ​​the aerial image falls inside the vector red line polygon. If so, it is determined that there is a spatial relationship. S32. Reverse containment detection: If S31 determines that there is no correlation, check whether any vertex of the vector red line polygon falls inside the geographic area of ​​the aerial image. If so, it is determined that there is a spatial correlation. S33. Polygon intersection detection: If S31 and S32 both determine that there is no correlation, check whether there is an intersection between the geographic area of ​​the aerial image and the boundary of the vector red line polygon. If so, it is determined that there is a spatial correlation.

10. The automatic selection method for oblique photogrammetric images oriented towards a target area according to claim 9, characterized in that, Before performing the multi-spatial topology relationship discrimination in step S3, the vector red line of the target area is expanded by a preset distance to generate a buffer polygon, which is used as the boundary for spatial topology relationship discrimination. Alternatively, in the corner point geodetic coordinate calculation in step S2, the ground plane is replaced with a digital elevation model (DEM), and the intersection of the direction vector V and the real ground surface represented by the DEM is calculated.

Images