Three-dimensional terrain modeling method and system

Abstract

The invention provides a three-dimensional terrain modeling method and system. The modeling method comprises the following steps: acquiring a plurality of aerial photos of a to-be-modeled region, wherein ground control points of the aerial photos are not marked; generating a first sparse point cloud set and a rough three-dimensional terrain model of the to-be-modeled region based on the aerial photo of each unmarked ground control point; automatically marking ground control points on each aerial photo based on the rough three-dimensional terrain model; generating a second sparse point cloud set of the to-be-modeled region based on the aerial photo of each marked ground control point; and generating a dense point cloud set and a fine three-dimensional terrain model of the to-be-modeled region based on the second sparse point cloud set. According to the modeling method provided by the invention, the marking speed is effectively improved while the marking precision of the ground control points is improved, and the efficiency of high-precision three-dimensional terrain modeling is remarkably improved on the whole.

Description

Technical Field

[0001] The present application belongs to the field of aerial photogrammetry technology, relates to three-dimensional modeling technology based on aerial images, and specifically provides a three-dimensional terrain modeling method and system. Background Art

[0002] The three-dimensional terrain modeling method based on aerial photogrammetry is a technology that uses photographic images and related data to reconstruct three-dimensional terrain models. This modeling method has the advantages of high precision, high efficiency and low cost. It has been widely used in geographic information systems (GIS), urban planning, land resources and major infrastructure project surveys, environment and natural disaster monitoring and other fields. With the development of drone technology, the application of three-dimensional modeling based on photogrammetry in the data collection process is becoming more and more common, making real-time and large-scale terrain modeling possible.

[0003] The existing three-dimensional terrain modeling methods based on aerial photogrammetry generally include the steps of obtaining a set of aerial pictures of the area to be modeled, matching and aerial triangulation of each aerial picture to establish a sparse point cloud of the area to be modeled, using a densification algorithm to obtain a dense point cloud of the area to be modeled and generating a three-dimensional terrain digital model of the area to be modeled. In order to improve the modeling accuracy, a number of ground control points (GCPs) can be arranged in the area to be modeled, and the captured ground control points can be marked on the aerial pictures, so as to provide geometric constraints for the iterative optimization of the point cloud coordinates in the subsequent aerial triangulation stage. Obviously, whether the position of the ground control points can be accurately calibrated on the aerial pictures will significantly affect the measurement and modeling results.

[0004] At present, the marking of ground control points on aerial images is mainly done manually, that is, manually selecting each ground control point on each aerial image and establishing an association between the same ground control point in different aerial images. This operation not only consumes a lot of time, but also significantly reduces the modeling speed. In addition, there are also methods for automatically identifying ground control points through texture information on aerial images. However, the existing algorithms do not establish a matching relationship between different aerial images when automatically identifying ground feature points. As a result, even if the images of each ground feature point can be extracted from each aerial image through texture features, the same ground feature points on different aerial images cannot be associated. Generally, it is necessary to manually specify the association between the ground control points on each aerial image, otherwise different ground control points on two or more aerial images may be mistakenly identified as the same one, resulting in an adverse effect on subsequent aerial triangulation measurement results. Summary of the invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the present application provides a three-dimensional terrain modeling method through an embodiment, and the method comprises the following steps:

[0006] S1, obtaining aerial images of several unmarked ground control points in the area to be modeled;

[0007] S2, generating a first sparse point cloud set and a rough three-dimensional terrain model of the area to be modeled based on the aerial pictures of each unmarked ground control point;

[0008] S3, automatically marking and associating ground control points on each aerial image based on the rough three-dimensional terrain model;

[0009] S4, generating a second sparse point cloud set of the area to be modeled based on the aerial images of each marked ground control point;

[0010] S5, generating a dense point cloud set and a fine three-dimensional terrain model of the area to be modeled based on the second sparse point cloud set.

[0011] Compared with the prior art, the three-dimensional terrain modeling method provided by the present application adds an operation of generating a sparse point cloud using aerial photos without marked ground control points to the conventional steps of three-dimensional terrain modeling based on aerial photogrammetry, and uses this operation to establish a rough three-dimensional terrain model, and then uses the rough three-dimensional terrain model to automatically mark and associate ground control points. Through the above steps, the association relationship between the same ground control points in different aerial photos can be automatically identified and accurately established without manual operation one by one, which effectively improves the marking speed while improving the marking accuracy of the ground control points, thereby significantly improving the efficiency of high-precision three-dimensional terrain modeling as a whole.

[0012] Further, step S2 includes the following steps:

[0013] S21, identifying multiple feature points in each aerial image;

[0014] S22, matching feature points in different aerial images;

[0015] S23, obtaining a first estimated value of a camera orientation element of each aerial image and a first estimated value of a three-dimensional coordinate of each feature point based on an unconstrained bundle adjustment;

[0016] S24, constructing a first sparse point cloud set based on the first estimated value of the three-dimensional coordinates of each feature point;

[0017] S25, establishing a rough three-dimensional terrain model of the area to be modeled based on the first sparse point cloud set.

[0018] Further, step S3 includes the following steps:

[0019] S31, selecting a ground control point as a target ground control point;

[0020] S32, selecting a reference image and several evaluation images corresponding to the target ground control point;

[0021] S33, determining at least one annotation point from the image corresponding to the target ground control point on the reference image;

[0022] S34, projecting the marked points onto the surface of the rough three-dimensional terrain model and intercepting a first projection facet;

[0023] S35, searching for the best position and best orientation of the first projection plane element;

[0024] S36, projecting an image in a reference window on the reference picture to a first projection plane element having an optimal position and an optimal orientation to obtain a second projection plane element;

[0025] S37, based on the expected image obtained by reprojecting the second projection surface element onto each of the to-be-marked pictures, searching and determining the imaging position corresponding to the target ground control point on each of the to-be-marked pictures.

[0026] Preferably, the similarity between the image of the target ground control point on the reference picture and its orthographic image is greater than a preset first similarity threshold, or the deviation is less than a preset first deviation threshold.

[0027] Preferably, the similarity between the image of the target ground control point on the evaluation picture and its orthographic image or its image on the reference picture is greater than a preset second similarity threshold, or the deviation is less than a preset second deviation threshold, wherein the first similarity threshold is greater than or equal to the second similarity threshold, and the first deviation threshold is less than or equal to the second deviation threshold.

[0028] Further, step S34 includes the following steps:

[0029] The marked point is projected onto the surface of the rough three-dimensional terrain model along a reference projection line corresponding to a reference image to obtain its projection point on the surface of the rough three-dimensional terrain, wherein the reference projection line corresponding to the reference image of the marked point is determined based on the camera orientation element of the reference image.

[0030] Further, step S35 includes the following steps:

[0031] S351, reprojecting the first projection surface element onto each evaluation image to obtain a reprojection window;

[0032] S352, determining the cross-correlation characteristics of the images in the reprojection windows of each evaluation picture;

[0033] S353, determining whether the mutual correlation feature satisfies the correlation criterion, if not, executing step S354 and returning to step S351, if satisfied, executing step S355;

[0034] S354, translating or rotating the first projection surface element along the reference projection line;

[0035] S355: Taking the current position and orientation of the first projection plane element as the optimal position and optimal orientation of the first projection plane element.

[0036] Further, step S37 includes cyclically executing the following steps until all the aerial pictures that need to be marked are processed:

[0037] S371, selecting an aerial picture of the area to be modeled as a picture to be marked;

[0038] S372, reprojecting the second projection bin onto the image to be labeled to obtain a desired image;

[0039] S373, extracting the edge of the desired image to construct a matching window;

[0040] S374, searching for the best position of the matching window on the image to be marked, thereby marking the target ground control point on the image to be marked.

[0041] Preferably, in the process of searching for the best position of the matching window on the image to be marked, the matching window is only translated on the image to be marked.

[0042] Further, step S4 includes the following steps:

[0043] S41, identifying multiple feature points in the aerial pictures of each marked ground control point and performing feature point matching;

[0044] S42, obtaining a second estimated value of a camera orientation element of each aerial image and a second estimated value of a three-dimensional coordinate of each feature point by a bundle adjustment with constraints, wherein the constraints include position constraints of marked ground control points;

[0045] S43: construct a second sparse point cloud set based on the second estimated value of the three-dimensional coordinates of each feature point.

[0046] The present application also provides a three-dimensional terrain modeling system, including a processor and a readable storage medium, wherein the readable storage medium stores an executable program, and when the executable program is executed by the processor, the aforementioned three-dimensional terrain modeling method can be implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1A flowchart of an existing method for three-dimensional terrain modeling based on aerial images;

[0048] Figure 2 A flowchart of a three-dimensional terrain modeling method provided according to an embodiment of the present application;

[0049] Figure 3A is a schematic diagram of an aerial picture of an area to be modeled in some embodiments;

[0050] Figure 3B Aerial photos are displayed in magnified form;

[0051] Figure 4 This is a flowchart of the implementation of step S2 provided according to an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the principle of iteratively estimating camera orientation elements and three-dimensional coordinates of feature points using an unconstrained bundle adjustment method according to an embodiment of the present application;

[0053] Figure 6 A flowchart for implementing the iterative estimation of camera orientation elements and three-dimensional coordinates of feature points using an unconstrained bundle adjustment method according to an embodiment of the present application;

[0054] Figure 7 This is a flowchart of the implementation of step S3 provided according to an embodiment of the present application;

[0055] Fig. 8A A schematic diagram of a reference image provided according to an embodiment of the present application;

[0056] Figure 8B A schematic diagram of an evaluation picture provided according to an embodiment of the present application;

[0057] Fig. 9 A schematic diagram of intercepting a first projection surface element on a rough three-dimensional model according to an embodiment of the present application;

[0058] Fig.10 A flowchart of searching for the optimal position and optimal orientation of a target ground control point according to an embodiment of the present application;

[0059] Fig.11 A schematic diagram of an operation of searching for an optimal position and an optimal orientation of a target ground control point according to an embodiment of the present application;

[0060] Fig.12 A schematic diagram showing how the NCC coefficient of an image in a reprojection window varies with the size of a first projection bin according to an embodiment of the present application;

[0061] Fig.13 is an implementation flow chart of step S37 provided according to an embodiment of the present application;

[0062] Fig.14 It is a schematic diagram of an operation of searching for the best position of a target ground control point matching window on a to-be-marked picture according to an embodiment of the present application;

[0063] Fig.15 It is a schematic diagram of the operation of searching for the best position of a matching window on a to-be-marked picture partially showing a target ground control point according to an embodiment of the present application;

[0064] Fig.16 A schematic diagram of a first sparse point cloud of an area to be modeled according to an embodiment of the present application;

[0065] Fig.17 A schematic diagram of a rough three-dimensional terrain model of an area to be modeled according to an embodiment of the present application;

[0066] Fig.18 A schematic diagram of a dense point cloud of an area to be modeled according to an embodiment of the present application;

[0067] Fig.19 A schematic diagram of a fine three-dimensional terrain model of an area to be modeled according to an embodiment of the present application;

[0068] Fig. 20 A schematic diagram of a three-dimensional model of a region to be modeled according to an embodiment of the present application;

[0069] Fig.21 Schematic diagram of the architecture of a three-dimensional terrain modeling system provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0070] Hereinafter, the present application will be further described based on preferred embodiments with reference to the accompanying drawings.

[0071] In addition, various components in the drawings are enlarged or reduced in size for ease of understanding, but this practice is not intended to limit the scope of protection of the present application.

[0072] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the products of the embodiments of the present application are usually placed when in use, it is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, in the description of the present application, in order to distinguish different units, the words first, second, etc. are used in this specification, but these are not limited by the order of manufacture, nor can they be understood as indicating or implying relative importance, and their names may be different in the detailed description and claims of the present application.

[0073] The vocabulary in this specification is used to illustrate the embodiments of the present application, but is not intended to limit the present application. It should also be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, an indirect connection through an intermediate medium, or a connection between the two elements. For those skilled in the art, the specific meanings of the above terms in this application can be specifically understood.

[0074] Figure 1 An existing three-dimensional terrain modeling method based on aerial photogrammetry is shown. The method first obtains a set of aerial pictures consisting of multiple aerial pictures that meet a certain overlap rate of the area to be modeled by aerial photography or other means, and then marks the ground control points (GCP) photographed on the aerial pictures. Further, by matching and aerial triangulation of each aerial picture, a sparse point cloud of the area to be modeled is obtained. Finally, a dense point cloud of the area to be modeled is obtained through a densification algorithm, and a three-dimensional terrain digital model of the area to be modeled is generated using the dense point cloud. In the above three-dimensional terrain modeling process, the ground control points serve as basic data to provide geometric constraints for iterative optimization of point cloud coordinates in the aerial triangulation stage. Obviously, whether the position of each ground control point can be accurately calibrated on the aerial picture and the same ground control points in different pictures can be accurately associated will significantly affect the measurement and modeling results.

[0075] The current mainstream method of manually marking ground control points requires manually selecting each ground control point on each aerial image and establishing an association between the same ground control point in different aerial images. This operation not only consumes a lot of time, but also significantly reduces the modeling speed.

[0076] In addition, there are also algorithms for automatically identifying ground control points through texture information on aerial images. However, the applicant found that the above-mentioned automatic recognition algorithm based on texture features of ground control point images has the following problems:

[0077] First, the same ground control point may have a large shape deformation and imaging area difference in aerial images of different perspectives. For example, the image of a ground control point that is square in one aerial image (photographed in front projection) may be compressed into a very thin rectangle in another aerial image (the shooting perspective is almost perpendicular to the normal direction of the ground control point). Since the texture information of the two is too different, neither calculating correlation nor relying on deep networks can solve this problem.

[0078] Secondly, when a ground control point is imaged at the edge of an aerial image, it may happen that the image only includes a partial image of the ground control point. In this case, the conventional image recognition method may fail to correctly identify the ground control point due to the lack of complete image information.

[0079] More importantly, the image of a ground control point in a certain aerial image may be incorrectly associated with the image of another ground control point in another aerial image for some reason. The reason for this problem is that when using, for example Figure 1 In the existing modeling method shown, when marking the ground control points, the camera orientation information of each aerial picture is not limited, resulting in matching based only on texture information. In this case, image distortion caused by perspective deformation or terrain structure factors in the shooting area may lead to matching errors. For example, there are some relatively similar buildings that are arranged periodically, causing the recognition algorithm to identify two different ground control points with different spatial positions but similar texture features in aerial pictures taken at two different locations as the same one.

[0080] It can be seen that if the matching relationship between different aerial images has not been established when the ground feature points are automatically identified, then even if the image of each ground feature point can be extracted from each aerial image through texture features, the same ground feature points on different aerial images cannot be associated. Generally, it is necessary to manually specify the association between the ground control points on each aerial image. Otherwise, different ground control points on two or more aerial images may be mistakenly identified as the same one, resulting in an adverse effect on the subsequent aerial triangulation measurement results.

[0081] The root cause of the above problems is that the existing automatic recognition algorithm lacks the necessary posture information when identifying ground control points. It is obviously impossible to fundamentally solve the problem simply by optimizing the feature extraction algorithm or the deep learning network structure, or increasing the training level of the deep learning network.

[0082] In order to solve the problems existing in the above-mentioned prior art, the present application improves the ground control point annotation and association method in the existing modeling process, thereby providing a new three-dimensional terrain modeling method. Figure 2 is a flow chart of a three-dimensional terrain modeling method provided according to an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0083] S1, obtaining aerial images of several unmarked ground control points in the area to be modeled;

[0084] S2, generating a first sparse point cloud set and a rough three-dimensional terrain model of the area to be modeled based on the aerial pictures of each unmarked ground control point;

[0085] S3, automatically marking and associating each ground control point on each aerial image based on the rough three-dimensional terrain model;

[0086] S4, generating a second sparse point cloud set of the area to be modeled based on the aerial images of each marked ground control point;

[0087] S5, generating a dense point cloud set and a fine three-dimensional terrain model of the area to be modeled based on the second sparse point cloud set.

[0088] By comparison Figure 1 and Figure 2 It can be seen that compared with the prior art, the three-dimensional terrain modeling method provided by the present application first uses aerial photos without marked ground control points (GCP) to generate a sparse point cloud in step S2, and uses this operation to establish a rough three-dimensional terrain model, and then uses the rough three-dimensional terrain model in step S3 to automatically mark and associate ground control points. Through this step, the association relationship between the same ground control points in different aerial photos can be automatically identified and accurately established without manual marking and association one by one, thereby providing accurate constraints for subsequent aerial photo registration and a second sparse point cloud generation operation through aerial triangulation.

[0089] The above steps are described in detail below with reference to the accompanying drawings.

[0090] <Get aerial images of the area to be modeled>

[0091] Step S1 is used to obtain an initial aerial picture of the area to be modeled, that is, an aerial picture without ground control point marking.

[0092] Figure 3A In a specific embodiment, several original aerial pictures obtained by aerial photography of an area to be modeled are shown. The aerial picture set comes from a collection of aerial pictures (Dort-ZECHE dataset) with a building as the designated area published by the International Society for Photogrammetry and Remote Sensing (ISPRS).

[0093] Figure 3B One of the aerial photos was enlarged and displayed. Figure 3B As shown in the red circles, in different aerial pictures obtained through aerial photography, there may be images of 0, 1, 2 or more ground control points.

[0094] In the process of aerial photogrammetry and 3D terrain modeling, ground control points can provide effective geometric constraints with their precise position information. Generally, they can be made of plates or soft fabrics, and their shape and size are determined according to the aerial photography altitude, shooting resolution, etc. (for example, a square of 50cm×50cm, or 1m×1m, etc.), and patterns with obvious texture features and high contrast are printed or painted on the surface.

[0095] Furthermore, before conducting aerial photography of the area to be modeled, a number of ground control points can be placed at a certain density on the ground, buildings or natural objects in the area to be modeled according to the established aerial photography plan, and the spatial position of each ground control point can be measured with high precision using a total station, GPS, etc.

[0096] After completing the above settings, you can carry out aerial photography of the area to be modeled according to the pre-established aerial photography plan. The image data obtained from aerial photography can be multiple aerial pictures or aerial videos taken continuously. The above aerial images generally need to be converted according to the requirements of subsequent data processing, such as unifying the image size, or converting continuous videos into the form of multiple aerial pictures. The number of aerial pictures is generally determined by factors such as the scale of the area to be modeled, the area covered by each aerial picture, and the overlap of adjacent aerial pictures. The overlap between consecutive aerial pictures is usually set between 60% and 80% to ensure the reliability of subsequent feature matching.

[0097] <Establish a rough 3D terrain model of the area to be modeled>

[0098] Step S2 is used to establish a rough three-dimensional terrain model of the area to be modeled. The rough three-dimensional terrain model is used to provide "indexes" and constraints for the imaging positions, shapes and texture features of each ground control point on different aerial images in the step S2 described later, so that the same ground control points can be identified and associated on different aerial images without manual one-by-one association.

[0099] Figure 4 The specific implementation process of step S2 in some preferred embodiments is shown as follows: Figure 4 As shown, the rough three-dimensional terrain model can be established by the following steps:

[0100] Step S21, identifying multiple feature points in each aerial picture;

[0101] Step S22, matching feature points in different aerial pictures;

[0102] Step S23, obtaining a first estimated value of a camera orientation element of each aerial image and a first estimated value of a three-dimensional coordinate of each feature point based on an unconstrained bundle adjustment;

[0103] Step S24, constructing a first sparse point cloud set based on the first estimated value of the three-dimensional coordinates of each feature point;

[0104] Step S25: establishing a rough three-dimensional terrain model of the area to be modeled based on the first sparse point cloud set.

[0105] Specifically, first in step S21, feature points can be extracted from each aerial picture by using a feature detection algorithm (such as SIFT, SURF or ORB, etc.); then in step S22, feature points of different aerial pictures can be matched. This step can use methods such as nearest neighbor search to match the same feature points in aerial pictures of different perspectives.

[0106] After completing the feature point matching of the aerial image, the bundle adjustment operation can be performed in step S23 to estimate the spatial position of each feature point. It should be emphasized that in the method provided in the present application, since the ground control points are not marked and associated on the aerial image before executing step S23, this step adopts the bundle free network adjustment that does not use position coordinates for constraints.

[0107] Unconstrained bundle adjustment refers to a method of estimating the camera orientation elements (including internal orientation elements such as the camera's focal length, principal point, distortion coefficient, and external orientation elements such as the camera's position and orientation) and the three-dimensional coordinates of each feature point of an aerial image without relying on external known control points.

[0108] Figure 5 and Figure 6 The principles and specific processes of iteratively estimating the three-dimensional coordinates of camera orientation elements and feature points using the unconstrained bundle adjustment method are respectively shown in some specific embodiments. Figure 6As shown, in step S231, the internal and external orientation elements of the camera can be initially estimated through the description document of the camera parameter specifications and the flight position and posture records of the aircraft during aerial photography, and then the three-dimensional coordinates of each feature point can be calculated based on the camera orientation elements and feature point matching results of each aerial picture through the triangulation method (step S232).

[0109] Then, in step S233, the first objective function value is calculated using the three-dimensional coordinates of each feature point and the camera orientation elements of each aerial picture (generally, the objective function of optical adjustment is a robust function of the sum of squares of mapping errors, that is, the error is minimized after each feature point is mapped to the aerial picture), and in step S234, it is determined whether the function value converges or the number of iterations reaches a preset upper limit. If the judgment result is no, the camera orientation elements are re-estimated and the three-dimensional coordinates of each feature point are recalculated through step S235, and then the first objective function value is recalculated. If the judgment result is yes, based on the current iteration result, the first estimated value of the camera orientation element of each aerial picture and the first estimated value of the three-dimensional coordinates of each feature point are determined (step S236).

[0110] After obtaining the first estimated value of the three-dimensional coordinates of each feature point, each feature point can be used as a point cloud point in step S24, and its position in the three-dimensional space can be set according to the first estimated value, thereby obtaining a first sparse point cloud set. Then in step S25, the above-mentioned feature points in the sparse point cloud set are used as grid points, and algorithms such as Delaunay triangulation are used to construct a rough three-dimensional terrain model of the area to be modeled. Generally, the rough three-dimensional terrain model constructed in this step is composed of a number of small patches, and the vertex of each patch is a feature point.

[0111] In some preferred embodiments, the camera orientation elements of each aerial image obtained through the above steps, the sparse point cloud set of the area to be modeled, and the rough three-dimensional terrain model are stored in a database for use in subsequent steps.

[0112] <Automatically mark and associate each ground control point>

[0113] It is particularly important to point out that in conventional three-dimensional terrain modeling methods, since the spatial resolution of the sparse point cloud composed of feature points is too low, the three-dimensional model generated therefrom is relatively rough. Therefore, the three-dimensional grid model is generally not generated directly from the sparse point cloud, but the sparse point cloud is densified and then the three-dimensional terrain is reconstructed using the dense point cloud. However, in the embodiments of the present application, since the surface of the rough three-dimensional terrain model reflects the approximate three-dimensional shape of the terrain of the area to be modeled, that is, the position and posture of each facet of its surface can roughly characterize the orientation and spatial coordinates of the area to be modeled at that location, it can be regarded as a reference for finding ground control points on each aerial image and establishing the correlation between the same ground control point on each aerial image, so that the ground control points are automatically marked and associated with the rough three-dimensional terrain model in step S3.

[0114] Figure 7 In some specific embodiments, the specific implementation steps of step S3 are shown, such as Figure 7 As shown, step S3 further includes the following steps:

[0115] Step S31, selecting a ground control point as a target ground control point;

[0116] Step S32, selecting a reference image and several evaluation images corresponding to the target ground control point;

[0117] Step S33, determining at least one annotation point from the image corresponding to the target ground control point on the reference image;

[0118] Step S34, projecting the marked points onto the surface of the rough three-dimensional terrain model and intercepting the first projection surface element;

[0119] Step S35, searching for the best position and best orientation of the first projection plane element;

[0120] Step S36, projecting an image in a reference window on the reference picture to a first projection plane element having an optimal position and an optimal orientation to obtain a second projection plane element;

[0121] Step S37 , based on the expected image obtained by reprojecting the second projection surface element onto each picture to be marked, searching and determining the imaging position corresponding to the target ground control point on each picture to be marked.

[0122] In some specific embodiments, when there are multiple ground control points that need to be automatically marked and associated, such as Figure 7 As shown, step S31 to step S37 may be executed cyclically until the operations for all ground control points that need to be automatically identified and associated are completed.

[0123] The implementation of step S3 is described in detail below with reference to the accompanying drawings.

[0124] First, in step S31, a ground control point is selected as the target ground control point that needs to be automatically marked and associated on each picture to be marked, and then a reference picture is selected based on the imaging conditions of the target ground control point on each aerial picture, and several evaluation pictures are selected around it.

[0125] In order to ensure the accuracy of subsequent search results, the reference image should be selected so that the target ground control point is photographed at an orthographic angle as much as possible. That is, the aerial image whose image corresponding to the target ground control point is close to its orthographic shape should be selected as the reference image. For example, when the actual shape of the target ground control point is a square, the aerial image that captures the target ground control point and whose imaging shape is basically a square should be selected from the aerial image collection.

[0126] In some preferred embodiments, the deviation or similarity can be used to evaluate the deviation or similarity between the shape of the image formed by a ground control point on the aerial picture and the shape it should have when photographed according to the orthographic projection perspective. For example, the Hausdorff distance, average distance or similarity ratio can be used to measure the geometric difference between the imaged image and the ideal square; for another example, the difference between the length of each side (or perimeter) of the image formed by the GCP on the aerial picture and the length of each side (or perimeter) of a standard square with the same area can be used as a deviation index:

[0127]

[0128] Among them, D is the deviation index, l i is the length of each side of the image formed by a GCP on the aerial photo, l square is the side length of a square of equal area to the image.

[0129] Obviously, a preset similarity threshold (first similarity threshold) or deviation threshold (first deviation threshold) can be set. When the similarity between the shape of the image formed by a ground control point on an aerial picture and the shape of its orthographic image is greater than the first similarity threshold, or the deviation is less than the first deviation threshold, the aerial picture is used as a reference picture with the ground control point as the target ground control point.

[0130] After determining the benchmark image of the target ground control point, several aerial images can be selected "around" the benchmark image as evaluation images. Here, "around" means that the evaluation images have a camera position and viewing angle close to those of the benchmark image. Since aerial images are generally numbered consecutively in the order of shooting, generally, the evaluation images in the aerial image collection are several adjacent images before and after the benchmark image.

[0131] Fig. 8A and Figure 8B Schematic diagrams of a reference image and surrounding evaluation images selected from a number of aerial images are respectively shown in a specific embodiment, after the target ground control point to be marked is determined.

[0132] like Fig. 8A As shown in , the evaluation image may include an image of the target ground control point taken at a near-front projection angle, or an image of the target ground control point taken at a more deviated angle than the reference image. Similarly, Figure 8B As shown, the evaluation pictures can also be screened according to the similarity or deviation between the shape of the image formed by the target ground control point on it and the shape of the orthographic image of the ground control point (or the shape of the image formed on the reference picture), that is, a second similarity threshold or a second deviation threshold can be preset, and only the aerial pictures whose similarity with the shape of the orthographic image of the target ground control point (or the shape of the image formed on the reference picture) is greater than the second similarity threshold or the deviation is less than the second deviation threshold are selected as evaluation pictures.

[0133] Preferably, the image formed by the target ground control point on the reference image is closer to its orthographic image than the image formed by the surrounding evaluation image. Therefore, in some preferred embodiments, the first similarity threshold is greater than or equal to the second similarity threshold; in other preferred embodiments, the first deviation threshold is less than or equal to the second deviation threshold.

[0134] In some preferred embodiments, the number of evaluation pictures is not less than 6. For example, 3 aerial pictures can be selected in sequence on both sides of the reference picture as 6 evaluation pictures. In other preferred embodiments, the number of evaluation pictures is not more than 10. For example, 5 aerial pictures can be selected in sequence on both sides of the reference picture as 10 evaluation pictures.

[0135] Step S33 is used to mark the annotation point corresponding to the target ground control point on the reference image. The marking in this step can be performed manually, for example, manually selecting a pixel in the image formed by the target ground control point on the reference image (preferably the pixel corresponding to the center of the target ground control point), and using the pixel as the annotation point of the target ground control point on the reference image; or, it can also be performed in a semi-automatic manner of human-computer interaction, for example, firstly manually defining a rough range, identifying the image of the ground control point therein through an automatic recognition algorithm and marking a pixel therein as a annotation point, and then manually adjusting the position of the annotation point.

[0136] It should be noted that although the operation of marking the target ground control point in step S33 occurs after the selection of the reference picture in step S32, the preferred principle for selecting the reference picture is whether a ground control point that is desired to be automatically marked can be imaged in an aerial picture in an orthographic or near-orthographic direction. The purpose is to facilitate accurate marking of the ground control point in the aerial picture. Therefore, for different target ground control points, different aerial pictures may need to be selected as their reference pictures.

[0137] After the target ground control points are accurately marked on the reference image, in step S34, Fig. 9 As shown, first, the marked points of the target ground control points are projected onto the rough three-dimensional terrain model of the area to be modeled along their projection lines corresponding to the reference image (in the embodiments of the present application, the projection lines corresponding to the marked points on the reference image are referred to as reference projection lines), wherein the reference projection lines of the marked points can be determined by the camera orientation elements of the reference image obtained in step S2 and stored in the database.

[0138] Camera orientation elements play an important role in photogrammetry, computer vision, image processing and other fields. They include the camera's intrinsic orientation elements and extrinsic orientation elements. The intrinsic orientation elements describe the camera's internal parameters, which are related to the camera's geometric structure and optical properties, mainly including focal length, principal point, radial distortion coefficient, tangential distortion coefficient, etc., which are used to characterize how to project from a three-dimensional space point to a two-dimensional image plane; the extrinsic orientation elements describe the camera's spatial position and orientation during shooting, mainly including the camera's three-dimensional position coordinates and rotation angles (Roll, Pitch, Yaw or the corresponding rotation matrix / quaternion), which are used to characterize the camera's position and posture relative to the world coordinate system.

[0139] refer to Figure 5 Once the camera orientation elements of an aerial image are determined through aerial triangulation, for a three-dimensional space point, the straight line from the camera center to this space point is the projection line of the space point corresponding to the aerial image. The intersection of this projection line and the aerial image is the position of its image on the aerial image. Therefore, the projection line shows how the space point is mapped to the image through the internal and external orientation elements of the camera.

[0140] In order to clearly indicate the direction of the projection operation, in an embodiment of the present application, the operation of projecting one or several elements (such as pixels, geometric features, or texture features) from an aerial image along their corresponding projection lines onto a three-dimensional terrain model in the form of a mesh patch of the area to be modeled is called projection; correspondingly, the operation of projecting the elements on the three-dimensional terrain model onto the aerial image along their projection lines is called reprojection.

[0141] As analyzed above, the rough 3D model obtained by unconstrained bundle adjustment can roughly reflect the 3D terrain of the area to be modeled, and the camera orientation element provides the direction of projection of each pixel on the reference image to its corresponding spatial point. Therefore, after projecting the annotated point of the target ground control point onto the surface of the rough 3D model according to its corresponding reference projection line on the reference image, a projection point will be obtained. The 3D coordinates of the projection point and the orientation of the plane on which it is located on the rough 3D model (i.e., the normal direction) represent the approximate spatial position and orientation of the target ground control point when it is arranged in the area to be modeled.

[0142] Then, if Fig. 9 As shown, continue with step S34 to intercept a surface element around the projection point on the plane where the projection point on the rough groove three-dimensional model is located. In the present application, the surface element is called the first projection surface element.

[0143] The first projection surface element can preferably be intercepted with the projection point as the center, or it can be adjusted accordingly according to the actual intersection of the planes constituting the rough three-dimensional model (for example, when the surface element intercepted with the projection point as the center happens to span two planes constituting the rough three-dimensional model, the first projection surface element can be appropriately offset to be in the same plane). It is only necessary to ensure that the projection point is included in the intercepted surface element.

[0144] The shape and size of the first projection surface element can be selected according to the scale of the area to be modeled, the resolution parameters of the aerial image, etc. In some preferred embodiments, the first projection surface element can be in the shape of a square, with the projection point as the center of the square. In other embodiments, the shape of the first projection surface element can also be set accordingly according to the actual shape of the target ground control point, such as a rectangle, a circle, etc.

[0145] In an embodiment of the present application, the first projection surface element is equivalent to a Mask, which constructs a "window" with a limited shape and size (number of pixels) based on the approximate position and orientation information of the target ground control point reflected by the projection point on the rough three-dimensional model. Furthermore, this can be used as the starting point for the search, and an iterative search can be performed in step S35 to obtain the best estimate of the position and orientation of the target ground control point.

[0146] refer to Fig.10 as well as Fig.11 In some preferred embodiments of the present application, the search for the optimal position and orientation of the target ground control point is performed by the following steps:

[0147] Step S351, reprojecting the first projection surface element onto each evaluation image to obtain a reprojection window;

[0148] Step S352, determining the cross-correlation characteristics of the images in the reprojection windows of each evaluation picture;

[0149] Step S353, determining whether the mutual correlation feature satisfies the correlation criterion, if not, executing step S354 and returning to step S351, if satisfied, executing step S355;

[0150] Step S354, translating or rotating the first projection surface element along the reference projection line;

[0151] Step S355: taking the current position and orientation of the first projection plane element as the optimal position and optimal orientation of the first projection plane element.

[0152] Specifically, in step S351, the first projection plane element is constructed according to its current position and orientation, and the projection lines of each point on it projecting onto each evaluation picture are constructed according to the camera orientation element information of each evaluation picture (determined in step S2), and reprojected onto each evaluation picture along the projection line corresponding to each evaluation picture. After reprojection to the evaluation picture, the area covered on the evaluation picture is as follows: Fig.11 The edge of this region, shown in red, constitutes the reprojection window.

[0153] Obviously, during the first reprojection operation, the intersection of the first projection element and the reference projection line is the projection point on the rough three-dimensional model, and the position and orientation of the first projection element are the position and orientation of the element intercepted in step S34.

[0154] refer to Fig.11The reprojection window represents the expected position, size and shape (i.e., the position of the reprojection window, the number of pixels covered, and the shape formed) of the target ground control point on the corresponding aerial image after it is photographed by the cameras at various positions when the target ground control point is in a certain alternative posture. Obviously, only the first projection facet that is consistent with the actual posture of the target ground control point, whose texture features in the reprojection window obtained by reprojecting it to each evaluation image, has the greatest correlation with the target ground control point.

[0155] Since the rough three-dimensional model has obtained the approximate position and posture of the target ground control point, using it to determine the initial position and posture of the first projection surface element can significantly reduce the amount of search calculations. Only a slight translation or rotation around the initial position and posture is required to accurately estimate the true position and posture of the target ground control point. At the same time, for any candidate position and posture of the first projection surface element, its position determines the position of the reprojection window on the evaluation image, and its orientation determines the shape and size of the reprojection window on the evaluation image. It can be seen that the texture features of the image in the reprojection window are also restricted and constrained by factors such as position, posture, shape, and area. Any reprojection window generated according to any position that deviates from the actual position and posture by more than a certain degree will cause the image it covers to deviate significantly from the actual imaging result of the target ground control point on the evaluation image in one or more of the position, shape, and area, and further cause a significant decrease in the correlation of the image texture in the reprojection window. Therefore, the above method can greatly improve the accuracy of estimating the actual position and posture of the target ground control point.

[0156] When evaluating the correlation of images in each reprojection window in step S352, an image correlation evaluation index known to those skilled in the art may be selected to calculate the correlation of images in the reprojection windows on any two evaluation pictures. For example, an optional correlation evaluation index is the Normalized Cross-Correlation (NCC), or other indicators for image correlation may be selected.

[0157] In some preferred embodiments, the reference picture is also used as the evaluation picture, that is, the first projection bin is also reprojected to the reference picture, and the image in its reprojection window is used to participate in the evaluation of the correlation.

[0158] After completing the correlation calculation of the images in each reprojection window on each evaluation picture, it is judged in step S353 whether the calculation result meets the correlation criterion, and step S354 or step S355 is selected to be executed according to the judgment result.

[0159] In some optional embodiments, the correlation of the images in each reprojection window may be firstly counted to obtain any one or more of the statistical characteristics such as the mean, median, maximum or minimum, and then determine whether it is greater than or equal to a preset correlation threshold. When the determination result is no, the first projection plane element is translated or rotated in step S354 (refer to Fig.11 , the translation or rotation of the first projection surface element is always performed with the intersection of the first projection surface element and the reference projection line as the movement base point, that is, the projection result of the annotation point of the target ground control point to its spatial point is always "bound" to the reference projection line, and the translation and rotation of it are always performed along the reference projection line), and then return to step S351 to re-project and recalculate the correlation. When the judgment result is yes, execute step S355, and use the position and orientation (normal direction) of the first projection surface element at this time as its optimal position and optimal orientation. The position and orientation will be regarded as the most likely estimated result of the actual position and orientation of the target control point in the area to be modeled, and provide effective constraints when searching and associating the target control point on the image to be marked later.

[0160] Fig.12 The figure shows how the NCC coefficient of an image in each reprojection window varies with the size of the first projection element in a specific embodiment. In the figure, the first projection elements are all square elements, and the horizontal axis represents the number of pixels contained in each side of the element.

[0161] like Fig.12 As shown, in the area where the first projection bin size is smaller, as the bin size increases, the NCC value generally shows a gradual upward trend, indicating that as the bin size increases, the matching accuracy also improves. However, as the size further increases, the NCC value no longer increases significantly, and even fluctuates (such as the fluctuation between 11×11 pixels and 17×17 pixels).

[0162] By analyzing the image texture features on each evaluation image, it can be seen that this correlation index no longer increases significantly after the pixel size increases to a certain extent, and even decreases. This indicates that as the number of pixels used for correlation calculation increases, other image noise may be introduced, resulting in a decrease in the contribution of the pixel size increase to the pose optimization, or even a negative impact. Therefore, the size of the first projection pixel cannot be too small, otherwise there will be a lack of sufficient information for evaluating the cross-correlation in the subsequent search process. Preferably, the lower limit of the size of the first projection pixel is 3×3 pixels; at the same time, the size of the first projection pixel does not need to be too large, because further increasing the size will not continue to improve the cross-correlation. In some preferred embodiments, the upper limit of the size of the first projection pixel can be determined by calculating the maximum value of the correlation statistics of the images in each reprojection window.

[0163] After determining the actual position and posture of the target ground control point, its actual position and posture can be used as a geometric constraint condition. Obviously, using this constraint condition can effectively limit the search for the target ground control point on the image to be marked to a specific area, thereby significantly improving the efficiency of automatic search. More importantly, using this geometric constraint, the image of the ground control point automatically searched and marked on the image to be marked must be the image of the target ground control point. That is to say, while automatically searching, the association of the same ground feature point on different aerial images is also completed, thereby fundamentally avoiding the problem of identifying the images of different ground control points on two aerial images as the same ground control point.

[0164] like Figure 7 As shown, searching for target ground control points on the image to be marked is achieved through steps S36 and S37.

[0165] Step S36 is used to generate a second projection surface element: the image edge corresponding to the target ground control point can be generated on the reference picture by manual operation or manual plus automatic algorithm interactive operation, so as to obtain a reference window; the image in the reference window is projected along the reference projection line onto the first projection surface element with the best position and the best orientation, so as to obtain the second projection surface element.

[0166] Step S37 is used to automatically search for target ground control points on each image to be marked. As analyzed above, since this step searches for the specified target ground control points on each image to be marked, it also establishes the association between the same ground control points on different aerial images.

[0167] In some specific embodiments, Fig.13 As shown, step S37 includes cyclically executing the following steps until all the aerial pictures that need to be marked are processed:

[0168] S371, selecting an aerial picture of the area to be modeled as a picture to be marked;

[0169] S372, reprojecting the second projection bin onto the image to be labeled to obtain a desired image;

[0170] S373, extracting the edge of the desired image to construct a matching window;

[0171] S374, searching for the best position of the matching window on the image to be marked, thereby marking the target ground control point on the image to be marked.

[0172] Specifically, in step S372, the second projection facet can be reprojected onto the image to be marked along the projection line corresponding to the image to be marked, so as to obtain its expected image on the image to be marked. Obviously, the position and orientation of the second projection facet are consistent with the first projection facet with the best position and orientation. The difference between the two is that the second projection facet adds the shape and texture feature information expected when the corresponding image of the target ground control point on the reference image is projected to the real position of the target ground control point. Therefore, when the multiple information with this specific position, orientation, shape and texture features is further reprojected onto the image to be marked, the expected image obtained can roughly limit the expected imaging position of the target ground control point on the image to be marked and the shape, size and texture of the expected image. The target ground control point is searched by using this information with strong constraints, which not only eliminates the possibility of identifying different ground control points in two aerial images as the same, but also only needs to translate and slide the matching window in a small area around the expected image, without the need for large-scale search and rotation operations in conventional image matching retrieval, which undoubtedly greatly improves the search speed.

[0173] Preferably, the search for the optimal position of the matching window in step S374 can be implemented using a least squares image matching algorithm, such as Fig.14 As shown, the matching window can be slid translationally on the image to be marked according to the set sliding step size, and the image in the matching window can be extracted, and the minimum error square calculation is performed on its texture features and the texture features of the expected image. The above process is repeated until the position with the minimum error is found. The image surrounded by the matching window at this position represents the position where the target ground control point is imaged, thereby completing the automatic marking of the target ground control point.

[0174] like Fig.14 As shown, when the desired image is completely inside the picture to be marked, the matching window can be constructed only by the edge of the extracted desired image; Fig.15 The illustrated embodiment further demonstrates the situation of searching for the best position of the matching window on the image to be marked that only partially displays the target ground control point. When the target ground control point is located at the edge of an aerial image, only a portion of it may be imaged on the aerial image. For the conventional automatic marking method based solely on texture information, it is obvious that the texture features used for search and comparison are missing, which makes recognition difficult. However, for the method of the present application, since the expected image is imaged at the edge of the aerial image according to its expected position, and only a portion of it is located on the aerial image, the edge of the expected image and the edge of the aerial image itself can be used to jointly determine the matching window, and search for its best position on the image to be marked. That is, through the method of the present application, it is possible to complete its identification and marking only through partial information of the target ground control point on the image to be marked.

[0175] Back to Figure 7 After completing the operations of steps S31 to S37 once, the automatic identification and association of a target ground control point is completed. After that, a ground control point can be reselected as the target ground control point, and then steps S31 to S37 can be re-executed. By repeatedly executing the above operations, the automatic marking and association of all or a specific number of ground control points can be completed.

[0176] <High-precision modeling of 3D terrain>

[0177] Back to Figure 2 After completing the operation of step S3, step S4 can be performed to generate a sparse point cloud set again using the aerial photos that have been marked and associated with the ground control points (i.e., a constrained bundle adjustment operation is performed under the constraints provided by the ground control points to obtain a second sparse point cloud).

[0178] In some specific embodiments, step S4 includes the following steps:

[0179] S41, identifying multiple feature points in the aerial pictures of each marked ground control point and performing feature point matching;

[0180] S42, obtaining a second estimated value of a camera orientation element of each aerial image and a second estimated value of a three-dimensional coordinate of each feature point by a bundle adjustment with constraints, wherein the constraints include position constraints of marked ground control points;

[0181] S43: construct a second sparse point cloud set based on the second estimated value of the three-dimensional coordinates of each feature point.

[0182] The main difference between step S4 and step S2 is that the bundle adjustment operation at this moment uses the position information of the ground control points that have been marked and associated as a constraint condition, specifically, because the true three-dimensional coordinates of each ground control point have been obtained by accurate measurement, the position of its imaging on each aerial picture has also been marked and associated by step S3, therefore, the position information of the ground control point can be used as a constraint condition, and the estimation result of the three-dimensional coordinate of each feature point is constantly optimized in the iterative estimation process of the bundle adjustment, thereby finally obtaining the second estimated value of the three-dimensional coordinate of each feature point and generating the second sparse point cloud.

[0183] After obtaining the second sparse point cloud, a dense point cloud can be generated through step S5 to obtain a fine three-dimensional terrain model of the area to be modeled. The specific implementations of the above steps S4 and S5 are already known to those skilled in the art and will not be repeated here.

[0184] <Specific Example>

[0185] This embodiment adopts Figure 3A and Figure 3B The aerial image collection shown is Figure 2 The method shown is used to model the buildings therein with high accuracy.

[0186] Fig.16 , Fig.17 The schematic diagrams of the three-dimensional sparse point cloud and the coarse grid model of the building generated during the implementation process are respectively shown. In the process of automatically marking and associating each ground control point, the size of the first projection surface element is 11×11 pixels. The following Table 1 lists the statistics of the marking results of 10 ground control points.

[0187] Table 1 Statistics of automatic marking results of ground control points

[0188]

[0189] It can be seen from Table 1 that when the reference window size is 15×15, a total of 482 points are successfully marked, with a success rate of about 67.3%, and the root mean square error between the coordinates of these 482 marked points and the true coordinates is 0.564 pixels; when the window size is 25×25, a total of 683 points are successfully marked, with a success rate of about 95.5%, and the root mean square error between the coordinates of these 683 marked points and their true coordinates is 0.549 pixels. The statistical results show that through the above-mentioned automatic marking and association steps, more than 700 ground control points to be marked can be automatically marked within 1 minute, and as the reference window size increases to 25×25 pixels, the marking success rate exceeds 95%. At the same time, the comprehensive accuracy of marking each ground control point reaches the sub-pixel level, proving that it is a comprehensive improvement in marking speed and marking accuracy compared to manual marking.

[0190] Fig.18 , Fig.19 The dense point cloud and fine 3D terrain model of the building are shown respectively. Fig. 20 To obtain the final 3D model after mapping the fine model, Figures 18 to 20 It can be seen that the three-dimensional terrain modeling method provided in the present application can efficiently and accurately complete the three-dimensional terrain modeling based on aerial photos.

[0191] Some embodiments of the present application also provide a three-dimensional terrain modeling system, such as Fig.21 As shown, the system includes a processor and a readable storage medium, wherein the readable storage medium stores an executable program, and when the executable program is executed by the processor, the aforementioned three-dimensional terrain modeling method can be implemented.

[0192] The above is a detailed introduction to the specific implementation methods of the present application. For those skilled in the art, several improvements and modifications may be made to the present application without departing from the principles of the present application. These improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A three-dimensional terrain modeling method, characterized in that: The following steps are involved: S1, obtaining aerial images of several unmarked ground control points in the area to be modeled; S2, generating a first sparse point cloud set and a rough three-dimensional terrain model of the area to be modeled based on the aerial pictures of each unmarked ground control point; S3, automatically marking and associating ground control points on each aerial image based on the rough three-dimensional terrain model; S4, generating a second sparse point cloud set of the area to be modeled based on the aerial images of each marked ground control point; S5, generating a dense point cloud set and a fine three-dimensional terrain model of the area to be modeled based on the second sparse point cloud set.

2. The three-dimensional terrain modeling method according to claim 1, characterized in that: Step S2 includes the following steps: S21, identifying multiple feature points in each aerial image; S22, matching feature points in different aerial images; S23, obtaining a first estimated value of a camera orientation element of each aerial image and a first estimated value of a three-dimensional coordinate of each feature point based on an unconstrained bundle adjustment; S24, constructing a first sparse point cloud set based on the first estimated value of the three-dimensional coordinates of each feature point; S25, establishing a rough three-dimensional terrain model of the area to be modeled based on the first sparse point cloud set.

3. The three-dimensional terrain modeling method according to claim 1, characterized in that: Step S3 includes the following steps: S31, selecting a ground control point as a target ground control point; S32, selecting a reference image and several evaluation images corresponding to the target ground control point; S33, determining at least one annotation point from the image corresponding to the target ground control point on the reference image; S34, projecting the marked points onto the surface of the rough three-dimensional terrain model and intercepting a first projection facet; S35, searching for the best position and best orientation of the first projection plane element; S36, projecting an image in a reference window on the reference picture to a first projection plane element having an optimal position and an optimal orientation to obtain a second projection plane element; S37, based on the expected image obtained by reprojecting the second projection surface element onto each of the to-be-marked pictures, searching and determining the imaging position corresponding to the target ground control point on each of the to-be-marked pictures.

4. The three-dimensional terrain modeling method according to claim 3, characterized in that: The similarity between the image of the target ground control point on the reference image and its orthographic image is greater than a preset first similarity threshold, or the deviation is less than a preset first deviation threshold; The similarity between the image formed by the target ground control point on the evaluation picture and its orthographic image or its image formed on the reference picture is greater than a preset second similarity threshold, or the deviation is less than a preset second deviation threshold, wherein the first similarity threshold is greater than or equal to the second similarity threshold, and the first deviation threshold is less than or equal to the second deviation threshold.

5. The three-dimensional terrain modeling method according to claim 3, characterized in that: Step S34 includes the following steps: The marked point is projected onto the surface of the rough three-dimensional terrain model along a reference projection line corresponding to a reference image to obtain its projection point on the surface of the rough three-dimensional terrain, wherein the reference projection line corresponding to the reference image of the marked point is determined based on the camera orientation element of the reference image.

6. The three-dimensional terrain modeling method according to claim 3, characterized in that: Step S35 includes the following steps: S351, reprojecting the first projection surface element onto each evaluation image to obtain a reprojection window; S352, determining the cross-correlation characteristics of the images in the reprojection windows of each evaluation picture; S353, determining whether the mutual correlation feature satisfies the correlation criterion, if not, executing step S354 and returning to step S351, if satisfied, executing step S355; S354, translating or rotating the first projection surface element along the reference projection line; S355: Taking the current position and orientation of the first projection plane element as the optimal position and optimal orientation of the first projection plane element.

7. The three-dimensional terrain modeling method according to claim 3, characterized in that: Step S37 includes cyclically executing the following steps until all the aerial images that need to be marked are processed: S371, selecting an aerial picture of the area to be modeled as a picture to be marked; S372, reprojecting the second projection bin onto the image to be labeled to obtain a desired image; S373, extracting the edge of the desired image to construct a matching window; S374, searching for the best position of the matching window on the image to be marked, thereby marking the target ground control point on the image to be marked.

8. The three-dimensional terrain modeling method according to claim 7, characterized in that: In the process of searching for the best position of the matching window on the image to be marked, the matching window is only translated on the image to be marked.

9. The three-dimensional terrain modeling method according to claim 1, characterized in that: Step S4 includes the following steps: S41, identifying multiple feature points in the aerial pictures of each marked ground control point and performing feature point matching; S42, obtaining a second estimated value of a camera orientation element of each aerial image and a second estimated value of a three-dimensional coordinate of each feature point by a bundle adjustment with constraints, wherein the constraints include position constraints of marked ground control points; S43: construct a second sparse point cloud set based on the second estimated value of the three-dimensional coordinates of each feature point.

10. A three-dimensional terrain modeling system, comprising a processor and a readable storage medium, characterized in that: The readable storage medium stores an executable program, and when the executable program is executed by the processor, the three-dimensional terrain modeling method according to claim 1 can be implemented.

Images