Unmanned aerial vehicle multi-constraint aerial survey image rare control point orientation method

Abstract

According to the unmanned aerial vehicle multi-constraint aerial survey image rare control point orientation method, an error equation of a plumb line is derived from a linear equation of the plumb line, and an error model of image bottom points is established on the basis of a collinear condition; solving exterior orientation elements of the image according to least square adjustment, and reducing the number of control points by a plumb line; constructing a block adjustment method under the constraint of a plumb line, obtaining exterior orientation elements of the images again according to the leveled model, then solving a transformation matrix between the exterior orientation elements before and after leveling, and obtaining exterior orientation elements of each leveled image; the method comprises the following steps: performing absolute orientation on a regional free network after leveling under a plumb line coordination condition, erecting an absolute orientation model under the plumb line coordination condition, realizing absolute orientation of five parameters only by two level height control points, and optimizing a sparse matrix to establish an unmanned aerial vehicle adjustment model under a plumb line constraint condition. And absolute orientation can be realized when parameters and control points are few, and the orientation efficiency and precision are high.

Description

Technical Field

[0001] The present application relates to a multi-constraint aerial survey sparse control point orientation method, and particularly to an unmanned aerial vehicle multi-constraint aerial survey image sparse control point orientation method, belonging to the technical field of aerial survey control point orientation. Background Technique

[0002] With the continuous development of pattern recognition and computer vision technologies, photogrammetry has also developed rapidly and increasingly relies on computers for automatic calculations. Aerial photogrammetry has become the main method for obtaining geographical information data. Subsequently, unmanned aerial vehicle (UAV) aerial survey has been widely recognized in the surveying and mapping industry, capable of acquiring ground images in a short time, providing more intuitive information for big data, and can be used in aspects such as urban construction, planning and scheduling, national conditions monitoring, disaster relief and rescue, tourism development, archaeological research, power line inspection, discovery of illegal buildings, discovery of illegal land use, highway patrol, terrain measurement of high and steep cliffs, slope monitoring across rivers, dangerous landslide areas, measurement of toxic areas, and dangerous high-voltage power facilities.

[0003] UAV aerial survey images need to be further processed, and new data analysis for aerial triangulation is required. Traditional aerial triangulation requires a large number of ground control points to be measured in the field. The strip method first establishes a free strip through relative orientation and model connection, uses the photogrammetric coordinates of points in the strip as observations, and determines the transformation parameters in a non-linear polynomial to incorporate the free network into the required ground coordinate system and minimize the sum of squares of discrepancies at common points. Traditional aerial survey has many disadvantages:

[0004] (1) In terms of flight: The flight attitude is unstable with a large rotation angle, and adjacent two images are generally prone to a particularly large change in the rotation angle under the influence of crosswind and unstable air currents; the image distortion is large, and the distortion at the edge can reach dozens of pixels or more; the camera image format is small and the overlap degree is irregular, etc.

[0005] (2) In terms of field control points: The layout of image control points is measured by field personnel, with high personnel costs, and it is difficult to enter mountainous areas and areas with dense vegetation; large-scale mapping requires a higher density of control points to ensure accuracy; affected by factors such as weather and environment, the time for laying out control points in the field is not easy to control, which is likely to cause project delays.

[0006] (3) In terms of the inner office: Aerial triangulation adjustment requires inner-office point piercing, which requires experience of operators. Unskilled operators are not easily able to pierce accurately, which affects the adjustment accuracy. A large amount of technical support and training are required. It takes several years to train good aerial triangulation personnel. In actual projects, it is impossible to layout field control points according to the layout requirements. To improve the accuracy of the results, both inner-office processing and technical support will consume a large amount of costs.

[0007] The current POS system still has a large number of errors and cannot be used in actual production applications. Therefore, it has good economic and practical value to use other methods to reduce the number of control points required in traditional regional network adjustment and improve the accuracy at low hardware costs.

[0008] The problems to be solved in the orientation of UAV aerial survey control points in the prior art and the key technical difficulties of this application include:

[0009] (1) The requirements for orientation accuracy in photogrammetry are getting higher and higher. The disadvantages of multi-image orientation at present include: 1) Traditional aerial triangulation requires more control points, which brings great difficulties to fieldwork; 2) POS-assisted aerial triangulation can appropriately reduce the number of control points, but to obtain high-precision information, high-precision equipment is required, the equipment cost is expensive, and the load is large; 3) Constrained by the geometric features of scenes such as buildings, the application scenarios are limited, and external model data is required; The orientation methods in the prior art have disadvantages and deficiencies, lacking orientation based on the plumb line, and it is impossible to obtain high orientation accuracy in the case of few control points, and the need for UAV aerial survey image orientation cannot be met with few control points.

[0010] (2) The main factor in photogrammetry for both field and office work is the measurement of control points. It has high labor intensity, high personnel cost, long cycle, and it is difficult to enter mountainous areas and areas with dense vegetation, making it difficult to meet the needs of actual applications. Moreover, large-scale mapping requires a higher density of control points to ensure accuracy. There are a large number of artificial buildings in the city, and their geometric features such as parallelism, symmetry, coplanarity, and perpendicularity are relatively obvious, providing good constraint conditions for reducing the number of control points. However, the prior art lacks multi-image orientation methods based on a large number of plumb lines in the city, analyzing the plumb line constraint conditions and POS constraint conditions, lacks the influence model of different constraint conditions on the image orientation results, does not analyze the relationship between the nadir point of the image and the exterior orientation elements of the image, and does not establish the error equation of the plumb line and the error model of the nadir point; lacks the absolute orientation elements and absolute orientation model of the regional free network after leveling, and cannot achieve absolute orientation when the parameters and control points are few, with low orientation efficiency and poor accuracy.

[0011] (3) To obtain better aerial triangulation results in UAV aerial surveying, a large number of control points need to be measured. However, the layout of control points is rather difficult. In some places, it is even impossible to access. Usually, field measurement cannot be completely carried out in accordance with the principles and requirements of control point layout, which affects the accuracy of image orientation results. Therefore, it is very important to find a method that can reduce or replace control points for image orientation. However, in cities, there are basically artificial buildings. Especially now, high-rise buildings are springing up everywhere. They contain a large number of obvious geometric features such as parallelism, symmetry, coplanarity, and perpendicularity, which can provide good constraint conditions for reducing the number of control points. However, the existing technologies have not effectively utilized them. There is a lack of an error model for the nadir point based on the collinearity condition, a method for regional network adjustment under the constraint of the plumb line, and a method for optimizing the sparse matrix to establish a UAV adjustment model under the constraint of the plumb line. Absolute orientation has a large dependence on the number of control points, with relatively low elevation accuracy; it has a great impact on planar accuracy and elevation accuracy. With a small number of control points, it is impossible to obtain good elevation accuracy; in actual engineering projects, the feasibility and reliability are relatively poor, and it is difficult to achieve good results in UAV aerial surveying control point orientation. It is even more difficult to provide strong support for sparse control point image orientation. Summary of the Invention

[0012] Starting from the theory of photogrammetric orientation, this application develops a multi-image orientation method under the cooperation of the plumb line, and analyzes adjustment models and calculation methods such as absolute orientation under the cooperation of the plumb line and UAV regional network adjustment under the cooperation of the plumb line. Combining data from Liupanshui, Pan County, Hong Kong, etc., it is verified that the number of control points can be reduced under the condition of plumb line cooperation, reducing the workload of field control point measurement personnel and interior control point piercing personnel, thereby improving production efficiency. It is confirmed that the cooperation of the plumb line reduces the dependence of absolute orientation on the number of control points, and the elevation accuracy is significantly improved after adding the cooperation of the plumb line; only a small number of control points are needed to obtain good elevation accuracy; on the basis of traditional image orientation, the cooperation of the plumb line or POS cooperation will improve the accuracy of the orientation result. The plumb line cooperative orientation of this application is verified for feasibility and reliability through three sets of actual engineering project data, achieving relatively good results and being relatively practical, providing strong support for further sparse control point image orientation.

[0013] To achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0014] The orientation method of UAV multi-constrained aerial survey images with sparse control points first analyzes the relationship between the nadir point and the image exterior orientation elements, obtains the nadir point based on several groups of images of the nadir point inside the film, establishes the error equation of the vertical line and the error equation of the nadir point, relatively orients the entire area to obtain the free network, then selects two images with longer baselines to establish a single model using the vertical line and level it, calculates the transformation matrix between the exterior orientation elements before and after the vertical line, and then transforms each image to level the entire area network, uses at least two control points to absolutely orient the free area network after leveling, and establishes the UAV adjustment model under the vertical line constraint condition;

[0015] 1) According to the vanishing point, the relationship between the perpendicular line, the nadir point, and the image exterior orientation elements is established. The error equation of the perpendicular line is derived from the straight line equation of the perpendicular line, and the error model of the nadir point is established based on the collinearity condition;

[0016] 2) Select multiple groups of images to establish a single model, then use the collinearity condition equation to obtain the error equation of the horizontal and vertical control points and the error equation of the vertical line, solve the exterior orientation elements of the image according to the least squares adjustment, and use the vertical line to reduce the number of control points;

[0017] 3) Construct a regional block adjustment method under the constraint of heavy vertical line. First, a single model is established. Then, the vector from the image bottom point to the photography center is parallel to the Z axis in space. The model is leveled in combination with the baseline of the image. The exterior orientation elements of the image are obtained again according to the leveled model. Then, the transformation matrix between the exterior orientation elements before and after leveling is obtained. Each image is transformed to obtain the exterior orientation elements of each image after leveling, and then the entire regional network is leveled.

[0018] 4) Absolute orientation of the regional free network after leveling under the condition of heavy vertical line coordination. After the regional network is leveled, the orientation parameters are reduced, and only 5 parameters are needed to achieve orientation. The absolute orientation model under the condition of heavy vertical line coordination is listed. Only two leveling and height control points are needed for the 5 parameters to achieve absolute precise orientation;

[0019] 5) Optimize the sparse matrix to establish the UAV adjustment model under the condition of heavy vertical line constraints.

[0020] Preferably, the perpendicular line constrains the orientation error model: the projections of all parallel lines in the aerial survey space on the image are straight lines, and strictly intersect at the vanishing point. The perpendicular line is a special type of parallel line in the space with the same direction and vertically downward. Its projection on the drone aerial photography image strictly intersects at the image bottom point, and also draws a perpendicular line downward through the projection center. The intersection of the perpendicular line and the image plane determines a straight line. The straight line equation of the perpendicular line is obtained according to the pixel coordinates of the projection of the spatial straight line at the two points on the image:

[0021] Ax+By+C=0 Formula 1

[0022] Based on a large number of plumb lines in urban UAV aerial surveys, read the pixel coordinates of two points near the two endpoints on the plumb line, find the straight-line equation of its projection on the image, assume that the plumb line passes through the projection center, and recognize that the nadir point corresponds to the infinite point. In the collinearity equation, X A = X S , Y A = Y S , Z A = ∞, then:

[0023]

[0024] The nadir point is the intersection point of all plumb lines, and the nadir point also passes through the straight line, then Ax n + By n + C = 0. After normalization: Since the straight line passes through the nadir point (x n , y n ), then there is:

[0025] Ax n + By n + C = 0 Equation 2

[0026] Perform linearization to obtain the observation error equation of the plumb line:

[0027]

[0028] In the above formula, x n (0) , y n (0) are calculated using the initial values of the exterior orientation elements by the nadir point formula, where:

[0029]

[0030] According to the calculation formula of the nadir point, the following two error equations of the nadir point are obtained:

[0031]

[0032] The plumb line constraint orientation error equation set for UAV aerial survey is obtained.

[0033] Preferably, the relative orientation of the plumb line collaborative connection method: The relative orientation of the connection method is based on the left image, and the relative orientation is carried out through the angular motion and linear motion of the image, and the orientation elements are In the relative orientation of multiple images, the successive relative orientation method is used. The orientation basis is the coplanarity condition equation. When performing image relative orientation in urban areas with a large number of plumb lines, according to the plumb line collaborative orientation, before the plumb line collaborative orientation, first perform relative orientation on the entire block network, and then level it. The leveling process first levels two images, obtains the leveling transformation matrix, and thus levels the entire block network. The detailed steps are as follows:

[0034] First step: Select the projections of more than two higher plumb lines on the left image, measure the pixel coordinates of the endpoints respectively, obtain the straight line equation of each plumb line, calculate the coordinates (x n , y n ) of the image nadir point, perform gross error rejection and redundant observations to improve the accuracy of the plumb line, and obtain the optimal result using the least squares method;

[0035] Second step: Assume that the values of the exterior orientation elements of the left image are all 0, and then according to the relationship of homologous image points of the left and right images, calculate the exterior orientation elements of the right image and the baseline vector between the left and right images;

[0036] Third step: Level the stereo pair using the position vector from the image nadir point of the left image to the projection center and the baseline vector;

[0037] The projection of the plumb line passing through the projection center of the image is the image nadir point. When leveling, the vector is parallel to the Z-axis, and then rotate the vectors and to the X, Y, and Z-axis directions through the following rotation matrix R;

[0038]

[0039] Fourth step: Recalculate the attitude angle elements ω and κ of the left image after leveling, and calculate the rotation matrix R corresponding to the image before and after leveling, and then perform transformation on all images after relative orientation to obtain the leveled block free network.

[0040] Preferably, for the multi-constraint orientation of the aerial block free network: Establish a free network model, obtain the absolute position of each model point, perform coordinate transformation. The conversion model usually used is the 7-parameter conversion model. In the conversion process of this application, the model in the block free network after the plumb line collaborative relative orientation is perpendicular to the ground, and only requires one rotation angle parameter K around the Z-axis, three position parameters ΔX, ΔY, ΔZ, and one scale parameter λ;

[0041]

[0042] It can be solved only by two horizontal and vertical control points, perform absolute orientation of coordinates, and then perform redundant observations to obtain the optimal parameter solution through adjustment, making the orientation result more accurate.

[0043] Preferably, the adjustment of the UAV regional network coordinated with the plumb line: Add the coordination of the plumb line on the basis of the original adjustment model. The coordinates of the control points are regarded as the true values, the exterior orientation elements include errors, the image point coordinates are regarded as the observed values, the coordinates of the encrypted points include errors, and the plumb line coordinates are used to coordinate the UAV regional network for orientation. Calculate the influence degree of the control points on the orientation accuracy, combine the error equation of the plumb line, and improve the UAV adjustment model to obtain the following form:

[0044]

[0045] Rewrite it in matrix form:

[0046]

[0047] where V xu , V c , V e , V v are the correction values of the image point coordinates, control point coordinates, exterior orientation element observed values, and nadir point observed values; A u is the exterior orientation element increment coefficient matrix, B u is the image point coordinate increment coefficient matrix, D is the coefficient matrix of the plumb line, L u , L c , L e , L v are the residuals of the image point coordinates;

[0048] According to the least squares adjustment, the matrix form is as follows:

[0049]

[0050] Furthermore, the correction values of the coordinates of each encrypted point and the exterior orientation elements of each image are obtained, and the values of the exterior orientation elements and the coordinates of the encrypted points are gradually iteratively calculated.

[0051] Preferably, based on a large number of plumb lines in the city, analyze the multi-image orientation method under the plumb line constraint condition and the POS constraint condition, and establish an influence model of different constraint conditions on the image orientation result;

[0052] 1) Analyze the relationship between the nadir point and the exterior orientation elements of the image, and establish the error equation of the plumb line and the error model of the nadir point;

[0053] 2) Use the line plumb line to level the single model established by the image, then calculate the transformation matrix between different exterior orientation elements before and after leveling, and then transform each image to achieve the leveling of the entire regional network;

[0054] 3) Establish the absolute orientation elements and absolute orientation model of the area free network after leveling. Only 5 parameters, namely two control points, are required to achieve absolute orientation under the condition of plumb line constraint.

[0055] 4) Establish an unmanned aerial vehicle adjustment model under the condition of plumb line constraint based on sparse matrix. The plumb line constraint improves the orientation elevation accuracy.

[0056] Compared with the prior art, the innovation points and advantages of this application are as follows:

[0057] (1) There are a large number of artificial buildings in the city, and their geometric features such as parallelism, symmetry, coplanarity, and perpendicularity are relatively obvious, providing good constraint conditions for reducing the number of control points. Based on a large number of plumb lines in the city, this application analyzes the multi-image orientation methods under the plumb line constraint condition and the POS constraint condition, and establishes an influence model of different constraint conditions on the image orientation result: 1) Analyze the relationship between the image nadir point and the exterior orientation elements of the image, and establish the error equation of the plumb line and the error model of the image nadir point; 2) Use the line plumb line to level the single model established by the image, and then find the transformation matrix between different exterior orientation elements before and after leveling, and then transform each image to achieve leveling of the entire area network. 3) Study the absolute orientation elements and absolute orientation model of the area free network after leveling. Under traditional conditions, 7 parameters are required, and only 5 parameters, namely two control points, are required to achieve absolute orientation under the plumb line constraint condition; 4) A sparse matrix is used to establish an unmanned aerial vehicle adjustment model under the plumb line constraint condition, and it is verified that the plumb line constraint can significantly improve the orientation elevation accuracy. In actual photogrammetry engineering projects, the elevation accuracy after aerotriangulation is much worse than the plane accuracy, thus reducing the overall quality of the product. However, the elevation accuracy can be improved under the condition of plumb line collaborative orientation. Therefore, the plumb line collaborative orientation method of this application has very important engineering practical value.

[0058] (2) Based on the fact that artificial buildings in cities contain a large number of obvious parallel, symmetrical, coplanar, and vertical geometric features, good constraints are provided to reduce the number of control points. The present application establishes the relationship between the vertical line, the image base point, and the image exterior orientation elements according to the vanishing point, and derives the error equation of the vertical line from the straight line equation of the vertical line, and establishes the error model of the image base point based on the collinearity condition; selects multiple groups of images to establish a single model, and then obtains the error equation of the horizontal height control point by the collinearity condition equation and the error equation of the vertical line, and solves the image exterior orientation elements according to the least squares adjustment, and reduces the number of control points by the vertical line; constructs a regional network adjustment method under the vertical line constraint, first establishes a single model, and then based on the collinearity condition equation, the error equation of the horizontal height control point is obtained and combined with the error equation of the vertical line. The vector from the bottom point of the image to the center of the photography is parallel to the Z axis in space. The model is leveled in combination with the baseline of the image. The exterior orientation elements of the image are obtained again according to the leveled model. Then the transformation matrix between the exterior orientation elements before and after leveling is calculated. Each image is transformed to obtain the exterior orientation elements of each image after leveling, and then the entire regional network is leveled; the regional free network is absolutely oriented after leveling under the condition of heavy vertical line coordination. The orientation parameters are reduced after the regional network is leveled. Only 5 parameters are needed to achieve orientation. The absolute orientation model under the condition of heavy vertical line coordination is listed. Only two horizontal height control points are needed for 5 parameters to achieve absolute precise orientation; the sparse matrix is ​​optimized to establish the UAV adjustment model under the condition of heavy vertical line constraints. Absolute orientation can be achieved even when there are few parameters and control points, and the orientation efficiency and accuracy are high.

[0059] (3) This application takes the photogrammetry orientation theory as the starting point, develops a multi-image orientation method under the condition of heavy plumb line coordination, and analyzes the adjustment models and settlement methods such as absolute orientation under heavy plumb line coordination and drone regional network adjustment under heavy plumb line coordination. It is verified by combining data from Liupanshui, Panxian, Hong Kong, etc. that the number of control points can be reduced under the condition of heavy plumb line coordination, which reduces the workload of field control point measurement personnel and internal control point puncture personnel, thereby improving production efficiency. It is confirmed that heavy plumb line coordination reduces the dependence of absolute orientation on the number of control points, and the addition of heavy plumb line coordination has significantly improved the elevation accuracy; only a small number of control points are needed to obtain better elevation accuracy; heavy plumb line coordination or POS coordination on the basis of traditional image orientation will improve the accuracy of the orientation results. The heavy plumb line coordinated orientation of this application has achieved relatively good results and is relatively practical by verifying the feasibility and reliability of three sets of actual engineering project data, providing strong support for further image orientation with sparse control points. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of obtaining image codes by constructing disordered image relations using global hashing improvements.

[0061] Figure 2 It is a schematic diagram of solving the relative directional elements.

[0062] Figure 3 It is a schematic diagram of the plumb line and the image nadir point.

[0063] Figure 4 It is a schematic diagram of the absolute orientation accuracy of the stereo model.

[0064] Figure 5 It is a schematic diagram of the distribution of control points with different numbers.

[0065] Figure 6 It is a schematic diagram of the distribution of check points for Experiment 1 of the coordinated multi-image orientation of the plumb line.

[0066] Figure 7 It is a schematic diagram of the accuracy of each point in the case of a certain number of control points in Experiment 1.

[0067] Figure 8 It is a schematic diagram of the mean square error of the check points in Experiment 1.

[0068] Figure 9 It is a map of a certain urban area in Hong Kong, China in the aerial image survey area of this experiment.

[0069] Figure 10 It is a schematic diagram of the experimental control points and check points. Detailed implementation manners

[0070] Next, in combination with the accompanying drawings, the technical solutions of the method for orienting sparse control points of UAV multi-constraint aerial survey images provided by this application will be further described, so that those skilled in the art can better understand this application and be able to implement it.

[0071] Urban 3D models and maps change relatively quickly. Generally, photogrammetry methods are used in production. First, aerial triangulation is carried out based on the digital images of UAV aerial surveys, and then models are generated. The first link is the most important. To obtain better aerial triangulation results, a large number of control points need to be measured. However, the layout of control points is relatively difficult, and in some places, it is even impossible to enter. Therefore, the field measurement usually cannot be completely carried out in accordance with the principles and requirements of the control point layout, but this also affects the accuracy of the image orientation results. Therefore, it is very important to find a method that can reduce or replace control points for image orientation, which has great economic and practical value. However, most of the buildings in the city are artificial buildings. Especially now, high-rise buildings are springing up everywhere. They contain a large number of obvious geometric features such as parallelism, symmetry, coplanarity, and perpendicularity, which can provide good constraint conditions for reducing the number of control points.

[0072] (1) Establish the relationship between the plumb line, the image nadir point, and the exterior orientation elements of the image according to the vanishing point. Derive the error equation of the plumb line from the straight-line equation of the plumb line, and establish the error model of the image nadir point based on the collinearity condition;

[0073] (2) Select multiple groups of images to establish a single model, then use the collinearity condition equation to obtain the error equation of the horizontal and vertical control points and the error equation of the vertical line. Use the least squares adjustment to solve the exterior orientation elements of the image, and use the vertical line to reduce the number of control points.

[0074] (3) Construct a regional block adjustment method under the constraint of heavy vertical line. First, a single model is established. Then, the vector from the image bottom point to the photography center is parallel to the Z axis in space. The model is leveled in combination with the baseline of the image. The exterior orientation elements of the image are obtained based on the leveled model. Then, the transformation matrix between the exterior orientation elements before and after leveling is calculated. Each image is transformed to obtain the exterior orientation elements of each image after leveling, and then the entire regional network is leveled.

[0075] (4) Absolute orientation of the regional free network after leveling under the condition of heavy vertical line coordination. After the regional network is leveled, the orientation parameters are reduced, and only 5 parameters are needed to achieve orientation. The absolute orientation model under the condition of heavy vertical line coordination is listed. The 5 parameters only need two leveling and height control points to achieve absolute precise orientation;

[0076] (5) Optimize the sparse matrix to establish the UAV adjustment model under the condition of heavy vertical line constraints.

[0077] The following conclusions can be drawn through this application method:

[0078] (1) The coordination of the heavy plumb line reduces the reliance of absolute orientation on the number of control points, and the elevation accuracy is significantly improved by adding the coordination of the heavy plumb line;

[0079] (2) When the vertical line is coordinated, the impact on the plane accuracy is small, but the impact on the elevation accuracy is large. Only a small number of control points are needed to obtain good elevation accuracy.

[0080] (3) The conclusions obtained by the traditional orientation method are also applicable in the case of heavy vertical line coordination. For example, the plane accuracy inside the regional network is slightly better than the plane accuracy around it. As the number of control points increases, both the plane accuracy and elevation accuracy of orientation increase.

[0081] (4) Based on traditional image orientation, performing vertical line coordination or POS coordination will improve the accuracy of the orientation results.

[0082] The coordinated orientation of the heavy vertical line in this application has been verified for its feasibility and reliability through three sets of actual engineering project data, and has achieved relatively good results and is relatively practical, providing strong support for further orientation with images of sparse control points.

[0083] 1. Orientation Error Model with Heavy Plumb Constraint

[0084] All parallel lines in the aerial survey space are projected as straight lines on the image and strictly intersect at the vanishing point. The plumb line is a special type of parallel line with the same direction and perpendicular downward in space. Its projection on the UAV aerial image strictly intersects at the nadir point. It is also to draw a plumb line downward through the projection center. The intersection point of the plumb line and the image plane, and two points determine a straight line. According to the pixel coordinates of the projections of two points of the space straight line on the image, the straight line equation of the plumb line is obtained:

[0085] Ax + By + C = 0 Equation 1

[0086] Based on a large number of plumb lines in urban UAV aerial surveys, read the pixel coordinates of two points near the two endpoints on the plumb line, and find the straight line equation of its projection on the image. Assume that the plumb line passes through the projection center, and the nadir point corresponds to the infinite point. In the collinearity equation, X A = X S , Y A = Y S , Z A = ∞, then:

[0087]

[0088] The nadir point is the intersection point of all plumb lines, and the nadir point also passes through the straight line. Then Ax n + By n + C = 0. After normalization: Since the straight line passes through the nadir point (x n , y n ), then there is:

[0089] Ax n + By n + C = 0 Equation 2

[0090] Perform linearization to obtain the observation error equation of the plumb line:

[0091]

[0092] In the above formula, x n (0) , y n (0) are calculated from the initial values of the exterior orientation elements using the nadir point formula, where:

[0093]

[0094]

[0095] According to the calculation formula of the nadir point, the following two error equations of the nadir point are obtained:

[0096]

[0097] Obtain the error equation system of the plumb line constraint orientation for UAV aerial survey.

[0098] II. Relative orientation of the plumb line collaborative connection method

[0099] The relative orientation of the connection method is based on the left image, and the relative orientation is carried out through the angular motion and linear motion of the image. The orientation elements are When using the relative orientation of the connection method in the relative orientation of multiple images, the orientation basis is the coplanarity condition equation. When carrying out image relative orientation in urban areas with a large number of plumb lines, according to the plumb line collaborative orientation, first carry out relative orientation on the entire regional network before the plumb line collaborative orientation, and then level it. In the leveling process, first level two images, obtain the leveling transformation matrix, and thus level the entire regional network. The detailed steps are as follows:

[0100] The first step: Select the projections of more than two higher plumb lines on the left image, measure the pixel coordinates of the endpoints respectively, obtain the straight line equation of each plumb line, and calculate the coordinates (x n , y n ) of the image nadir point, carry out gross error rejection and redundant observations to improve the accuracy of the plumb line, and use the least squares method to obtain the optimal result;

[0101] The second step: Assume that the values of the exterior orientation elements of the left image are all 0, and then according to the relationship of homologous image points between the left and right images, calculate the exterior orientation elements of the right image and the baseline vector between the left and right images;

[0102] The third step: Level the stereo pair by using the position vector from the image nadir point of the left image to the projection center and the baseline vector; such as Figure 1 ;

[0103] The projection of the plumb line passing through the projection center of the image is the nadir point. When leveling, the vector is parallel to the Z-axis, and then rotate the vectors and to the X, Y, and Z-axis directions through the following rotation matrix R;

[0104]

[0105] The fourth step: Recalculate the attitude angle elements ω and κ of the left image after leveling, and calculate the rotation matrix R corresponding to the image before and after leveling, and then transform all the images after relative orientation to obtain the leveled regional free network.

[0106] III. Multi-constraint orientation of the aerial photography regional free network

[0107] A free network model is established to obtain the absolute position of each model point, and coordinate transformation is carried out. The commonly used transformation model is a 7-parameter transformation model. During the transformation process of this application, the model in the regional free network after the relative orientation coordinated by the plumb line is perpendicular to the ground, and only one angular parameter K for rotation around the Z axis, three position parameters ΔX, ΔY, ΔZ, and one scale parameter λ are required;

[0108]

[0109] It can be solved only through two horizontal and height control points, coordinate absolute orientation is carried out, and redundant observations are performed. The optimal parameter solution is obtained through adjustment to make the orientation result more accurate.

[0110] IV. UAV regional network adjustment coordinated by the plumb line

[0111] After absolute orientation, the result is optimized. In this application, plumb line coordination is added to the original adjustment model to make the orientation result more accurate. The coordinates of the control points are regarded as true values, the exterior orientation elements include errors, the image point coordinates are regarded as observed values, the coordinates of the encrypted points include errors, and the plumb line coordinates are used to coordinate the UAV regional network adjustment orientation. Calculate the influence degree of the control points on the orientation accuracy, and combine the error equation of the plumb line to improve the UAV adjustment model to obtain the following form:

[0112]

[0113] Rewrite it in matrix form:

[0114]

[0115] where V xu , V c , V e , V v are the correction values of the image point coordinates, control point coordinates, exterior orientation element observed values, and nadir point observed values; A u is the exterior orientation element increment coefficient matrix, B u is the image point coordinate increment coefficient matrix, D is the plumb line coefficient matrix, L u , L c , L e , L v are the image point coordinate residuals;

[0116] According to the least squares adjustment, the matrix form is as follows:

[0117]

[0118] Furthermore, the correction values of the coordinates of each encrypted point and the exterior orientation elements of each image are obtained, and the values of the exterior orientation elements and the coordinates of the encrypted points are gradually iteratively calculated.

[0119] V. Orientation Experiments and Analyses in Multiple Situations

[0120] (I) Absolute Orientation in Collaboration with the Plumb Line

[0121] 1. Experiment Overview

[0122] This experiment used 54 aerial images of urban areas in Liupanshui, Guizhou. It was equipped with a SONY DSC - RX1R camera, with a total of 5 flight strips, a pixel size of 6000*4000, and a focal length of 35mm. Since the actual number of control points was small and could not meet the number of control points required for the experiment, first, a small number of field control points were used with the commercial software smart3D to perform aerial triangulation calculations on the images and measure the coordinates of a large number of encrypted points on the model, which were used as the control points and check points required for this experiment.

[0123] 2. Experiment Description

[0124] This experiment mainly verified that the number of control points would be reduced when performing absolute orientation in collaboration with the plumb line, and performed accuracy analysis on the orientation in collaboration with the plumb line. Since the original images would contain distortions due to materials, operations, etc. during the production and use of the camera, it was necessary to correct the distortions of the images before the experiment. First, free network adjustment was performed on all images to obtain the exterior orientation elements of each image in its free coordinate system. Then, some of the image groups were selected to form stereo pairs, and then two groups of experiments were carried out according to the following plan;

[0125] The first group: Use two control points and the plumb line;

[0126] The second group: Use three control points;

[0127] The specific steps are as follows:

[0128] Step 1: Measure more than 5 pairs of corresponding point image plane coordinates on the left and right images respectively. According to the image plane coordinates of these corresponding image points, perform relative orientation and solve the relative orientation elements. The process is as Figure 2 ;

[0129] Step 2: Solve the model coordinates of the control points according to the given image plane coordinates of the control points;

[0130] Step 3: Measure multiple plumb lines on the left image. According to the least squares method, find the optimal value of the image nadir point, and obtain the direction vector from the image nadir point to the projection center. Place the left image flat according to the fact that the direction vector is parallel to the Z - axis; Figure 3 are the plumb line and the image nadir point;

[0131] Step 4: Solve the absolute orientation elements of the stereo model according to the ground measurement coordinates and model coordinates of the control points.

[0132] 3. Experiment Results and Analyses

[0133] Through Figure 4 It can be seen that the plumb line collaboration will reduce the dependence of absolute orientation on the number of control points. Absolute orientation requires solving 7 parameters and theoretically requires at least 7 equations to be established. Traditional control generally requires actual measurement of three control points, while adding a plumb line for collaboration only requires two horizontal control points to achieve relative orientation. The orientation accuracy has little difference from that in the case of three control points, and the elevation accuracy is significantly improved by adding a plumb line for collaboration.

[0134] (2) Plumb line collaborative multi-image orientation experiment 1

[0135] 1. Experiment overview

[0136] In a certain urban area of Pan County, a single-lens SONY DSC-RXIR camera was used to take 345 images. The photo size is 6000*4000, and the focal length is 35mm. Downward and oblique images were taken respectively. There are 6 field precise control points and 7 check points.

[0137] 2. Experiment description

[0138] To fully prove the orientation effects under different constraint conditions, the following four groups of experiments were carried out respectively: (1) Traditional orientation, without adding plumb line and POS constraints; (2) Only adding plumb line constraints; (3) Only adding POS constraints; (4) Adding POS and plumb line constraints;

[0139] Based on the above several groups of experiments, different control points were added respectively for comparison on the basis of different orientation methods to verify the orientation effects achieved under different numbers of control points and different constraints. Since the experimental data is large in this experiment and the orientation effects of two control points and the plumb line are not very ideal, the selected numbers of control points are 3, 4, 5, and 6, and the distribution is as Figure 5 ;

[0140] The distribution of check points is as Figure 6 : First, the relative orientation of the connection method is carried out to obtain the exterior orientation elements of each image under free coordinates, then a free network model is established, and the free network is leveled through the nadir points of the images. Five absolute orientation elements are obtained according to the ground coordinates of the horizontal control points, and the free network model is absolutely oriented.

[0141] 3. Experimental results and analysis

[0142] Through the experiment, the mean square errors of the orientation control points and check points of various constraint methods in the case of 4 control points are as Figure 7 , and the mean square errors of the check points obtained in the case of various control points are as Figure 8 .

[0143] In the experiment, by analyzing the situation of all checkpoints, it can be seen that checkpoint No. 59 is located in the middle of the area and has good control effect. Control point No. 76 is in the lower right corner, far from the control points, and the control effect is poor. The following conclusions are obtained from the above data:

[0144] (1) In the case of the cooperation of the plumb line, the influence on the plane accuracy is small, and the influence on the elevation accuracy is large;

[0145] (2) In the case of the cooperation of the plumb line, the accuracies of the checkpoints under different distributions in the area will also be very different;

[0146] (3) The plane accuracy inside the block adjustment network is slightly better than that of the periphery;

[0147] (4) When the plumb line is added, the elevation accuracies of the vast majority of checkpoints tend to be stable, and the control effect of the plumb line elevation will be significantly improved, and the result is ideal; It can be obtained from the plane accuracy and elevation accuracy diagrams of checkpoint No. 59:

[0148] (1) In the case of the cooperation of the plumb line, the influence on the plane accuracy is small, and the influence on the elevation accuracy is large;

[0149] (2) As the number of control points increases, the traditional orientation method and the orientation method with the cooperation of POS will make the elevation accuracy gradually better. However, in the case of the cooperation of the plumb line, only a small number of control points are needed to obtain better elevation accuracy;

[0150] Adding the plumb line has little influence on the plane accuracy. It mainly improves the elevation accuracy. The POS cooperative adjustment has significant influence on both the plane and elevation accuracies. Precise POS data can significantly improve the orientation accuracy. However, to obtain higher-precision POS information, the cost is high. Therefore, the plumb line can be added for cooperation under appropriate circumstances.

[0151] (3) Multi-image orientation experiment two with the cooperation of the plumb line

[0152] 1. Experiment overview

[0153] This experiment uses aerial image data such as Figure 9 , and the survey area is located in a certain urban area of Hong Kong, China. There are many buildings in this area and most of them are high-rise buildings, so it is convenient to extract the plumb line. The whole survey area is flat. The image flight overlap is 80%, and the side overlap is 75%. 651 downward-looking images are selected for the experiment, with 5 control points and 7 checkpoints.

[0154] 2. Experiment description

[0155] The grouping situation of this experiment is the same as that of the previous experiment. For the orientation effects under different constraint conditions, four groups of experiments are carried out respectively:

[0156] (1) Traditional orientation, without adding the plumb line and POS constraints; (2) Based on traditional orientation, only add the plumb line for constraint and analyze the orientation results; (3) Add POS constraints for analysis; (4) Joint constraints of POS and plumb line;

[0157] Since the number of control points in this group of experiments is small, 5 control points are used in each of the above groups of experiments. The distribution of control points and check points is as Figure 10 .

[0158] 3. Experimental Results and Analysis

[0159] In this experiment, the number of control points remains unchanged, and only the orientation accuracy under different constraint conditions is analyzed. Through the collation and analysis of the experimental results, the error results of control points under various constraint conditions are obtained:

[0160] (1) Among the above four orientation methods, adding the plumb line or POS collaboration on the basis of traditional image orientation can improve the accuracy of the orientation results;

[0161] (2) The planar accuracy of the plumb line collaborative orientation is not very different from that of the traditional orientation method;

[0162] (3) The influence of the plumb line collaborative orientation on the elevation accuracy is obvious, and the nadir point has a good control effect on the elevation control.

[0163] The experiment first uses the data of Liupanshui, Guizhou to conduct an absolute orientation experiment on the image under the condition of plumb line collaboration. One group uses two control points and the plumb line, and the other group uses three control points. Through multiple comparison experiments, it can be seen that the number of control points can be reduced under the condition of plumb line collaboration. Use the data of Pan County and a certain area in Hong Kong, China to conduct multi-image orientation experiments, and conduct unconstrained, plumb line constraint, POS constraint, and joint constraint experiments of plumb line and POS respectively, verifying that the plumb line constraint can significantly improve the elevation accuracy of the orientation.

Claims

1. A method for orienting sparse control points of UAV multi-constraint aerial survey images, characterized in that, First, the relationship between the nadir point and the image exterior orientation elements is analyzed, and the nadir point is obtained based on several groups of images of the nadir point inside the film. The error equations of the heavy plumb line and the nadir point are established, and the free network is relatively oriented for the entire area. Then, two images with longer baselines are selected to establish a single model using the vertical plumb line and level them. The transformation matrix between the exterior orientation elements before and after the heavy plumb line is calculated, and then each image is transformed to level the entire regional network. At least two control points are used to absolutely orient the free regional network after leveling, and a UAV adjustment model under the heavy plumb line constraint is established. 1) According to the vanishing point, the relationship between the perpendicular line, the nadir point, and the image exterior orientation elements is established. The error equation of the perpendicular line is derived from the straight line equation of the perpendicular line, and the error model of the nadir point is established based on the collinearity condition; 2) Select multiple groups of images to establish a single model, then use the collinearity condition equation to obtain the error equation of the horizontal and vertical control points and the error equation of the vertical line. Solve the exterior orientation elements of the image according to the least squares adjustment, and use the vertical line to reduce the number of control points. 3) Construct a regional block adjustment method under the constraint of heavy vertical line. First, a single model is established. Then, the vector from the image bottom point to the photography center is parallel to the Z axis in space. The model is leveled in combination with the baseline of the image. The exterior orientation elements of the image are obtained based on the leveled model. Then, the transformation matrix between the exterior orientation elements before and after leveling is obtained. Each image is transformed to obtain the exterior orientation elements of each image after leveling, and then the entire regional network is leveled. 4) Absolute orientation of the regional free network after leveling under the condition of heavy vertical line coordination. After the regional network is leveled, the orientation parameters are reduced, and only 5 parameters are needed to achieve orientation. The absolute orientation model under the condition of heavy vertical line coordination is listed. Only two leveling and height control points are needed for the 5 parameters to achieve absolute precise orientation; 5) Optimize the sparse matrix to establish the UAV adjustment model under the condition of heavy vertical line constraints.

2. The method for orienting sparse control points of UAV multi-constraint aerial survey images according to claim 1, wherein Orientation error model with bi-vertical constraint: The projections of all parallel lines in the aerial survey space on the image are straight lines, and they strictly intersect at the vanishing point. Bi-vertical lines are special parallel lines with the same direction in space and vertically downward. Their projections on the drone aerial photography images strictly intersect at the image base point, and bi-vertical lines are also drawn downward through the projection center. The intersection of the bi-vertical line and the image plane determines a straight line. The linear equation of the bi-vertical line is obtained based on the pixel coordinates of the projections of the spatial straight line at two points on the image: Ax+By+C=0 Formula 1 Based on a large number of vertical lines in urban drone aerial surveys, the pixel coordinates of two points near the two end points on the vertical line are read, and the equation of the line projected on the image is calculated. Assuming that the vertical line passes through the center of photography, the image base point corresponds to the point at infinity. In the collinear equation, X A =X S ,Y A =Y S ,Z A =∞, then: The image nadir point is the intersection of all plumb lines. The image nadir point also passes through the line, so Ax n +By n +C = 0. After normalization: Since the line passes through the image nadir point (x n , y n ), then we have: Ax n +By n +C = 0 Equation 2 After linearization, we get the observation error equation of the plumb line: The above formula x n (0), y n (0) is calculated using the initial values of the exterior orientation elements by the image bottom point formula, where: According to the calculation formula of the nadir point, the error equations of the following two nadir points are obtained: The equation group of orientation error of UAV aerial survey with heavy vertical line constraint is obtained.

3. The method for orienting sparse control points of UAV multi-constraint aerial survey images according to claim 1, wherein, Relative orientation of the plumb line collaborative connection method: The relative orientation of the connection method is based on the left image, and the relative orientation is carried out through the angular movement and linear movement of the image. The orientation elements are The connection method relative orientation is used in the relative orientation of multiple images. The orientation basis is the coplanarity condition equation. When performing image relative orientation in urban areas with a large number of plumb lines, according to the plumb line collaborative orientation, first perform relative orientation on the entire regional network before the plumb line collaborative orientation, and then level it. In the leveling process, first level two images, calculate the leveling transformation matrix, and thus level the entire regional network. The detailed steps are as follows: Step 1: Select the projections of two or more higher vertical lines on the left image, measure the pixel coordinates of the endpoints respectively, obtain the linear equation of each vertical line, calculate the coordinates of the nadir point (x n , y n ), perform gross error rejection and redundant observations to improve the accuracy of the vertical lines, and use the least squares method to obtain the optimal result; Step 2: Assume that the values ​​of the exterior orientation elements of the left image are all 0, and then calculate the exterior orientation elements of the right image and the baseline vectors of the left and right images based on the relationship between the same-name image points of the left and right images; Step 3: Use the position vector from the bottom point of the left image to the camera station and the baseline vector to relatively level the stereo. The projection of the plumb line passing through the image station point is the nadir point. When the leveling vector is parallel to the Z-axis, and then the vectors and are rotated to the X, Y, and Z-axis directions through the following rotation matrix R; Step 4: Recalculate the attitude angle elements of the left image after leveling ω and κ, calculate the rotation matrix R corresponding to the images before and after leveling, and then transform all the images after relative orientation to obtain a free network of the area after leveling.

4. The method for multi-constraint aerial survey image sparse control point orientation of the unmanned aerial vehicle according to claim 1, wherein Multi-constraint orientation of the free network in the aerial photography area: Establish a free network model, obtain the absolute position of each model point, and perform coordinate transformation. The commonly used transformation model is the 7-parameter transformation model. During the transformation process of this application, the models in the regional free network after the relative orientation coordinated by the plumb line are perpendicular to the ground. Only one angular parameter K for rotation around the Z axis, three position parameters ΔX, ΔY, ΔZ, and one scale parameter λ are required; It can be solved only through two horizontal and vertical control points, perform absolute orientation of coordinates, and then perform redundant observations to obtain the optimal parameter solution through adjustment, making the orientation result more accurate.

5. The method for orienting sparse control points of UAV multi-constraint aerial survey images according to claim 1, wherein Plumb line coordinated UAV block adjustment: Add plumb line coordination on the basis of the original adjustment model. The coordinates of the control points are regarded as true values, the exterior orientation elements include errors, the image point coordinates are regarded as observed values, the coordinates of the encrypted points include errors, and the plumb line coordinated UAV block adjustment is oriented. Calculate the influence degree of the control points on the orientation accuracy, and combine the error equation of the plumb line to improve the UAV adjustment model to obtain the following form: Rewrite it in matrix form: where V xu , V c , V e , V v are the corrected values of image point coordinates, control point coordinates, observed values of exterior orientation elements, and corrected values of nadir point observations; A u is the exterior orientation element increment coefficient matrix, B u is the image point coordinate increment coefficient matrix, D is the coefficient matrix of the plumb line, L u , L c , L e , L v are the residuals of image point coordinates; According to the least squares adjustment, the matrix form is as follows: Furthermore, obtain the correction values of the coordinates of each encrypted point and the exterior orientation elements of each image, and gradually iterate to find the values of the exterior orientation elements and the coordinates of the encrypted points.

6. The method for multi-constraint aerial survey image sparse control point orientation of an unmanned aerial vehicle according to claim 1, characterized in that Based on a large number of plumb lines in the city, analyze the multi-image orientation methods under the plumb line constraint condition and the POS constraint condition, and establish an influence model of different constraint conditions on the image orientation result; 1) Analyze the relationship between the image nadir point and the exterior orientation elements of the image, and establish the error equation of the plumb line and the error model of the image nadir point; 2) Use the line plumb line to level the single model established by the image, then find the transformation matrix between different exterior orientation elements before and after leveling, and then transform each image to achieve the leveling of the entire block; 3) Establish the absolute orientation elements and absolute orientation model of the regional free network after leveling. Only 5 parameters, that is, two control points, are required to achieve absolute orientation under the plumb line constraint; 4) Based on the sparse matrix, establish a UAV adjustment model under the plumb line constraint condition, and the plumb line constraint improves the orientation elevation accuracy.

Images