Bundle adjustment method and system for camera and object point joint parallax angle parameterization

Abstract

The application discloses a camera and object point combined parallax angle parameterized bundle adjustment method and system, comprising the following steps: 1) obtaining remote sensing images by aerial photography of a measurement area, extracting and matching homonymous image point observation values of all remote sensing images; 2) arranging two prior anchor points to form a baseline and establish an angle reference frame; 3) uniformly parameterizing a camera pose, a photographic center position and a three-dimensional connection point position by a direction angle, an elevation angle and a parallax angle; 4) constructing a collinear observation equation to perform bundle adjustment calculation and jointly obtain the camera pose, the photographic center coordinates and the ground point coordinates. On the basis of parallax angle parameterization of the three-dimensional connection point, the photographic center and the camera pose are further uniformly incorporated into the angle parameterization system, so that the ill-conditioned and numerical instability problems under small parallax and weak control conditions are effectively improved, and the convergence and stability of the adjustment calculation are improved. The application is suitable for regional network adjustment of aerospace remote sensing images.

Description

Technical Field

[0001] This invention belongs to the field of photogrammetry technology, and particularly relates to a bundle adjustment method and system for parallax angle parameterization of camera and object point. Background Technology

[0002] With the development of new technologies in aerospace digital photogrammetry and remote sensing, remote sensing image processing technology is widely used in fields such as topographic mapping and geological disaster monitoring, and also has important applications in agricultural production and environmental governance. Among these applications, obtaining high-precision location information of ground features using high-resolution remote sensing images is crucial, as it affects the accuracy, quality, and usability of the final product.

[0003] Depending on the imaging distance, photogrammetry can be divided into close-range photogrammetry, aerial photogrammetry, and space photogrammetry. While image acquisition and data processing methods for close-range photogrammetry have developed rapidly, they are primarily based on Cartesian coordinate systems. When encountering poor observation conditions, low-quality corresponding points, or small parallax angles, the calculation results are often significantly affected.

[0004] However, aerospace photogrammetry involves large imaging distances and often very small parallax angles. In the process of solving the problem in a rectangular coordinate system, there is often an excessive number of iterations, resulting in low accuracy or even divergence in the solution results. This does not meet the actual production needs for rapidly acquiring ground information using remote sensing images.

[0005] Aerial triangulation can obtain the coordinates of ground-based densification points. Relatively mature aerial triangulation methods include strip-based area network adjustment, independent model adjustment, and bundle adjustment. Among these, bundle adjustment is one of the most rigorous aerial triangulation methods in photogrammetry. It uses the collinearity equation as the adjustment equation, and the observed values ​​are the original coordinates of corresponding image points. By rotating and translating the bundle in space, the optimal intersection of corresponding rays in the image pair model is achieved to obtain the coordinates of the three-dimensional tie points. Then, ground control points are used to normalize the solution into the global coordinate system. This method can comprehensively optimize the camera pose and the three-dimensional coordinates of densification points, obtaining a rigorous three-dimensional tie point coordinate solution, and is widely used in close-range and conventional photogrammetry.

[0006] As image resolution increases and the survey area expands, the relative increment values ​​of the three-dimensional coordinates of feature points expressed by the rectangular coordinate system vary greatly. This leads to ill-conditioned or near-singularity in the normal equation coefficient matrix of bundle adjustment. Furthermore, the matrix size increases exponentially with the size of the survey area and the number of image points. Therefore, the error equation coefficient matrix may be a large-scale sparse matrix of tens or hundreds of thousands of orders, posing a significant challenge to efficient image processing.

[0007] To improve the convergence of the classical bundle adjustment model, the industry often uses a large number of ground control points or more accurate camera exterior orientation elements to compensate. This approach can provide better initial values ​​for nonlinear optimization problems and increase the inconsistency of the normal equations, but it does not change the root cause of numerical instability under small parallax conditions from the perspective of variable expression. Existing methods propose parallax angle parameterization only for 3D connection points, so that variables strongly related to depth participate in optimization in the form of angles, thereby alleviating the numerical sensitivity of depth direction in Cartesian coordinates.

[0008] However, the aforementioned methods mostly only parameterize the disparity angles of the 3D connection points, while the camera center still participates in the optimization as a Cartesian coordinate unknown. Because the camera center and the 3D connection points are strongly coupled in the collinear equations, the "meter-scale" coordinates of the camera center and the "angle / pixel-scale" coordinates caused by image point observations are mixed, easily leading to inconsistent variable scales. Consequently, the condition number of the normal equations is still mainly dominated by the sub-blocks related to the camera center. Especially in scenarios with small disparity, weak control, or sparse control points, the observability of the camera center along the baseline extension direction weakens, easily leading to near-degree-of-freedom drift and propagating instability throughout the network. This results in slow iterative convergence, limited steady-state improvement, or even divergence.

[0009] Therefore, there is a need for a method that can uniformly parameterize the camera pose, the position of the photography center, and the position of the 3D connection point within the same angular reference frame and use them for joint solution of bundle adjustment, so as to reduce the risk of ill-conditioned problems and improve the steady state of the solution from the perspective of variable expression. Summary of the Invention

[0010] To address the shortcomings of existing technologies, the present invention aims to provide a bundle adjustment method and system for joint parallax angle parameterization of camera and object points. Under the baseline constraint formed by two prior anchor points, the camera pose, the position of the photography center, and the position of the three-dimensional connection point are uniformly incorporated into the angular parameter space expression, and the rigorous coordinates of the photography center and the coordinates of the three-dimensional connection point are obtained by joint solution through bundle adjustment.

[0011] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A bundle adjustment method with joint parallax angle parameterization of camera and object points includes the following steps: Acquire remote sensing images of the target area, and extract and match the observation values ​​of image points with the same name in all remote sensing images; An angular reference frame is established based on the matched image point observations and the prior anchor points of two known spatial coordinates. In the aforementioned angle reference frame, the camera attitude, the position of the photography center, and the position of the three-dimensional connection point are uniformly parameterized using the orientation angle, elevation angle, and parallax angle. Based on the parameterized camera pose, image center position, and 3D tie point position, a collinear observation equation is constructed and solved by bundle adjustment and joint iterative optimization to obtain the optimized image center coordinates and 3D tie point coordinates.

[0012] Furthermore, the two prior anchor points include a primary anchor point and a secondary anchor point. The angular reference frame is established with the primary anchor point as the origin and the baseline vector pointing from the primary anchor point to the secondary anchor point as the reference axis.

[0013] Furthermore, the parameterization of the camera center position is determined in the following manner: Based on the i The direction angle of the photography center With elevation angle Determine the first i The unit direction vector of each photography center is ; Based on the baseline formed by two prior anchor points, the first... i Unit direction vector of the camera center and the i Parallax angle of the photography center The geometric relationship of the triangle is used to derive the distance scale from the principal anchor point using the sine theorem. .

[0014] Furthermore, the first i A photography center Distance scale to the main anchor point The calculation formula is:

[0015] No. i A photography center The formula for calculating coordinates is:

[0016] in, Baseline length; The coordinates of the primary anchor point; For the first i Parallax angle of a photography center; For the first i Furthermore, the angle between the observation direction of each photography center and the baseline direction is further determined by the parameterization of the three-dimensional connection points in the following manner: Based on the j The orientation angle of each three-dimensional connection point With elevation angle Determine the first j The unit direction vector of each three-dimensional connection point is ; Based on the baseline formed by two prior anchor points, the first... jUnit direction vector of each three-dimensional connection point and the j parallax angle of three-dimensional connection points The geometric relationship of the triangle is used to derive the distance scale from the principal anchor point using the sine theorem. .

[0017] Furthermore, the first j Three-dimensional connection points Distance scale to the main anchor point The calculation formula is:

[0018] No. j Coordinates of the three-dimensional connection points for:

[0019] in, Baseline length; The coordinates of the primary anchor point; for j Parallax angle of three-dimensional connection points; for j The angle between the observation direction of each three-dimensional connection point and the baseline direction.

[0020] Furthermore, the camera pose parameterization is determined in the following way: For the i The camera pose of the image is determined using the pose angle. Perform parameterization and generate the first... i pose rotation matrix of the image .

[0021] Furthermore, constructing the collinear observation equations and performing bundle adjustment includes: For the i The first image j Given a point with the same name, its observation coordinates are... Through camera intrinsic parameter matrix Normalization yields the unit view direction vector in the camera coordinate system. ; The first j Coordinates of the three-dimensional connection points With the i Coordinates of the photography center The coordinate difference vector in the anchor point coordinate system, through the first... i pose rotation matrix of the image Transform to camera coordinate system to obtain object direction vector ; Construct collinear residual vectors ; The objective function is to perform nonlinear optimization to solve the problem by minimizing the sum of squares of collinear residuals of all image points with the same name.

[0022] Furthermore, before the iterative optimization solution, a parameter initialization step is also included: Based on the initial estimated coordinates of the camera center and the estimated coordinates of the three-dimensional points in space, and combined with the prior anchor point coordinates and baseline vector, the initial values ​​of the orientation angle, elevation angle and parallax angle corresponding to each camera center and each three-dimensional point in space are calculated in reverse. The initial values ​​of the camera attitude angles are provided by external measurement data or image relative orientation results.

[0023] On the other hand, the present invention provides a bundle adjustment system with joint parallax angle parameterization of camera and object point, comprising: Image point processing module: It is used to acquire remote sensing images of the target area, extract and match the observation values ​​of image points with the same name in all remote sensing images; Reference frame building module: It is used to build an angular reference frame based on the matched image point observations and the prior anchor points of two known spatial coordinates. Joint parameterization module: It is used to uniformly parameterize the camera attitude, the position of the photography center and the position of the three-dimensional connection point using the orientation angle, elevation angle and parallax angle in the angle reference frame; Adjustment solution module: It is used to construct collinear observation equations based on parameterized camera pose, image center position and 3D tie point position, and perform bundle adjustment joint iterative optimization to obtain optimized image center coordinates and 3D tie point coordinates.

[0024] Compared with the prior art, the present invention has the following beneficial effects: 1. Under the constraint of two prior anchor point baselines, this invention expresses the parallax angle parameterizedly for the photography center and the three-dimensional connection point in a unified manner, so that the depth scale is derived from the baseline geometric relationship, thereby reducing the sensitivity to depth instability under small parallax conditions.

[0025] 2. This invention unifies the camera pose, the center of photography, and the three-dimensional connection points in the angle domain parameter space for joint solution, reducing the scale inconsistency problem caused by the mixing of angle and length quantities, and improving the numerical stability of the adjustment process.

[0026] 3. This invention is applicable to weak control scenarios such as regional network adjustment of aerospace remote sensing images, and can obtain more stable photo center coordinates and rigorous three-dimensional connection point coordinate calculation results under conditions with fewer control points. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 This is a flowchart of an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the geometric relationship between the two prior anchor points, the photography center, and the three-dimensional connection point in an embodiment of the present invention. Figure 3 This is a schematic diagram of the parallax angle parameterization in an embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0031] like Figure 1 As shown, this embodiment provides a bundle adjustment method with joint parallax angle parameterization of camera and object points, including the following steps: Step S1: Acquire remote sensing images of the target area, extract and match the observation values ​​of the same image points in all remote sensing images; Step S2: Establish an angle reference frame based on the matched image point observations and the prior anchor points of two known spatial coordinates; Step S3: In the aforementioned angle reference frame, the camera attitude, the position of the photography center, and the position of the three-dimensional connection point are uniformly parameterized using the orientation angle, elevation angle, and parallax angle. Step S4: Based on the parameterized camera pose, camera center position, and 3D connection point position, construct the collinear observation equation and perform bundle adjustment and joint iterative optimization to obtain the optimized camera center coordinates and 3D connection point coordinates.

[0032] In step S1 of this embodiment, the imaging sensor fixed on the remote sensing platform is used to take pictures of the target area according to the prescribed route to obtain a series of remote sensing images, and the internal orientation parameters of the camera are obtained or the internal parameters are calibrated.

[0033] In step S2 of this embodiment, the original remote sensing image is first downsampled or scaled. Then, a certain number of seed points are extracted from the sampled image. The initial spatial relationship between the images is calculated using the correspondence between the seed points between two adjacent images. Next, the size of the overlapping area between the two images is calculated, and the overlapping area is divided into blocks using a block-segmentation algorithm to obtain one-to-one small image pairs. Finally, all observation values ​​of the same image points in the image pairs are extracted and matched, and the relationship between the observation value pairs of the same image points is constructed to obtain the image coordinate observation set of the same image points (image points of the same spatial three-dimensional connection point on different images).

[0034] In step S3 of this embodiment, as shown in Figure 1, two prior anchor points are set up as the main anchor point and the secondary anchor point. The main anchor point serves as the reference origin, and the line connecting the main and secondary anchor points forms a baseline as the reference axis. The baseline direction serves as the starting direction of the direction angle, and counterclockwise rotation is the positive direction, thus establishing a spatial angle reference frame. The two prior anchor points can be obtained through ground control surveying or GPS / GNSS / RTK surveying, or control points with known coordinates within the survey area can be directly selected as anchor points. Their "prior" property means that the anchor point coordinates are known before adjustment and remain fixed during the adjustment iteration. To ensure geometric strength, it is preferable to select two points with a larger baseline length that cover the survey area as the anchor point pair, and to ensure that the anchor point coordinate accuracy meets the preset requirements (for example, their coordinate standard deviation is significantly less than the accuracy of the three-dimensional connection point to be determined or less than 1 / 5 of the ground distance corresponding to one pixel). 1 / 10).

[0035] Let the coordinates of the main anchor point be... The coordinates of the secondary anchor point are Define the baseline vector and baseline length as follows:

[0036] in, For the main anchor point Pointing to secondary anchor point The baseline vector, This is the corresponding baseline length scalar, used to provide a scale reference.

[0037] In an angular reference frame, any spatial point can be determined by its orientation angle. f Elevation angle i Determine the unit direction vector, and further combine it with the disparity angle. oh The scale is derived from the baseline geometry, thus obtaining the rectangular coordinates.

[0038] The unit direction vector corresponding to the direction angle and elevation angle is defined as follows:

[0039] Based on the definition of the unit direction vector corresponding to the above orientation angle and elevation angle, joint parallax angle parameterization is performed: the camera center and the three-dimensional connection point are given a unified angle expression and the scale is derived.

[0040] (I) Photo Center Parametric For the first i There is a photography center, and its angular parameters are denoted as . Its unit direction vector is The angle between it and the baseline direction is defined as:

[0041] in Defined as "the angle between the two lines of sight pointing from the center of the photograph to the two prior anchor points"; by , With baseline length The triangular relationship formed is used to derive the distance scale from the camera center to the main anchor point using the sine theorem. :

[0042] The derivation process is as follows: In triangle △ In the middle, the baseline length is The angle at the center of the photography is The included angle at the main anchor point is Then the included angle at the secondary anchor point is By the Law of Sines, we have

[0043] again Substitute and you will get

[0044] Then the first i A photography center The formula for calculating coordinates is:

[0045] in, Baseline length; The coordinates of the main anchor point.

[0046] (ii) Parameterization of 3D connection points (also known as ground points or space points) For the first j There is a photography center, and its angular parameters are denoted as . Its unit direction vector is The angle between it and the baseline direction is defined as:

[0047] in Defined as "the angle between two lines of sight pointing from a three-dimensional connection point to two prior anchor points"; by , With baseline length The triangular relationship formed is used to derive the distance scale from the camera center to the main anchor point using the sine theorem. :

[0048] Similarly, in triangle △ middle, host, By the Law of Sines, we have

[0049] again Substitute and you will get

[0050] This will scale the location of the three-dimensional connection points. From known baselines With angle measurement Precisely determine No. j Coordinates of the three-dimensional connection points for:

[0051] in, Baseline length; The coordinates of the main anchor point.

[0052] (III) Camera attitude angle parameterization For the i The camera pose of the image is determined using the pose angle. Perform parameterization and generate the first... i pose rotation matrix of the image This is used for establishing subsequent collinear observation equations; in the adjustment solution, , and They are solved jointly as a single set of unknowns.

[0053] In step S4 of this embodiment, constructing the collinear observation equations and performing bundle adjustment includes: For the i The first image j The observed coordinates of three image points with the same name in the image coordinate system (pixel coordinate system or equivalent image plane coordinate system). Construct homogeneous image point vectors and utilize the camera intrinsic parameter matrix. Normalization yields the unit view direction vector in the camera coordinate system. :

[0054] in, These are the homogeneous coordinates of the image point; This is the camera intrinsic parameter matrix, which includes parameters such as focal length, principal point, and pixel scale. The line-of-sight vector is normalized by the intrinsic parameters. It represents the ray direction from the camera center to the image point in the camera coordinate system. It only reflects the direction information and does not include the distance scale, so there is no need to transform to the anchor point coordinate system.

[0055] Based on the collinearity condition, the photography center Image point corresponding to the line of sight and spatial point All three are collinear. Because... As already expressed in the camera coordinate system, to establish collinearity constraints in the same coordinate system, the spatial direction vector in the anchor point (survey area) coordinate system is... via attitude rotation matrix Transform to the camera coordinate system to obtain the object-side direction vector. The two direction vectors should satisfy:

[0056] in, For the first i The attitude rotation matrix of the image represents the rotation transformation from the anchor point (survey area) coordinate system to the camera coordinate system; and The coordinates of the photographic center and the spatial point are given in the anchor point (survey area) coordinate system; the symbol " "" indicates that the two vectors differ by a non-zero scale factor. (Depth scale), meaning that the two directions are consistent, thus satisfying the collinearity condition.

[0057] To eliminate the scale factor, a cross product can be used to construct scale-independent collinear residuals:

[0058] when The collinearity condition is satisfied at the same time.

[0059] Construct a least-squares objective function using observations of all image points with the same name:

[0060] in, Indicates the first i A set of corresponding points observed in the image.

[0061] The objective function is solved iteratively: in each iteration, the residuals are linearized, the normal equation is constructed and the unknown corrections are solved, and the attitude angles and various angle parameters are updated until the convergence condition is met, at which point the set of camera center coordinates is output. With the set of coordinates of the three-dimensional connection points .

[0062] Since the center of the photograph and the three-dimensional connection point are obtained by mapping angle parameters respectively, the adjustment iteration is mainly carried out in the angle domain, making the main unknowns more consistent in terms of dimensions, thereby reducing the scale inconsistency problem caused by the traditional "mixing of angle and length quantities" and improving the numerical stability of the normal equation.

[0063] Example 1 In a set of simulated regional networks, several images are set to form a weakly controlled, small parallax imaging condition, and sub-pixel-level random observation noise is added to the images. Two high-precision prior anchor points are used as baseline constraints, and the solution is obtained using both traditional bundle adjustment in rectangular coordinate parameter space and the combined parallax angle parameterized bundle adjustment of this invention. Simulations show that when the parallax angle is small and the control points are sparse, traditional methods are prone to unstable iteration step sizes, slow convergence, and even divergence; while in this invention, because the imaging center and 3D connection points are uniformly expressed in the angle domain, the iteration converges more easily, and the reprojection residuals can stably converge to the sub-pixel level.

[0064] Example 2 In the context of regional network adjustment for aerospace / airborne remote sensing imagery, two known coordinate control points within the survey area are selected as prior anchor points. A joint parallax angle parameterization model is then established based on corresponding points from multiple imagery images for adjustment calculations. In practice, this demonstrates that even with a reduced number of control points or lower initial value accuracy, a stable convergence process can still be maintained, and the calculated coordinates of the photogrammetric center and 3D tie points are consistent with those obtained using the rigorous bundle adjustment method. This improves the stability and engineering usability of the regional network adjustment.

[0065] The above embodiments also include low-sensitivity error analysis; the low-sensitivity error is illustrated below through a comparative analysis of error propagation.

[0066] To compare the error amplification phenomenon of depth / scale variables in Cartesian coordinate parameter space, consider using the baseline B The intersection triangle formed by the parallax angle. The distance from the spatial point to the anchor point can be obtained using the sine theorem. satisfy:

[0067] in The angle between the observation direction and the baseline direction. oh Let be the parallax angle of a point in space.

[0068] Regarding the above formula By taking the derivative, the exact first-order sensitivity can be obtained:

[0069] As can be seen from the above formula, when hour, The term will increase dramatically (which can be approximated at extremely small angles). This means that a given angular observation error of the same magnitude leads to a significant amplification of errors in distance scale / depth-related quantities, resulting in weak observability in the depth direction of rectangular coordinates and ill-conditioned normal equations. It should be emphasized that the above derivation is based on the exact sine theorem and is only used to explain the numerical characteristics of the rectangular coordinate parameter space; the parameterization and solution of this invention are also based on the exact relation of the sine theorem and do not rely on any approximations.

[0070] For the angle parameterization in this invention, the main unknowns in the adjustment are directly angle variables (including the orientation angle between the camera center and the 3D connection point, the elevation angle and the parallax angle, and the camera attitude angle), and their value range is naturally limited (e.g., parallax angle). Direction angle Elevation angle (All measurements are in radians). During iterative updates, the angle correction remains within a controllable range, thus preventing the Cartesian depth variable from being affected by small parallax. Amplified numerical risks.

[0071] More specifically, this invention represents the location of the photography center and the 3D connection point as "main anchor point + distance scale × unit direction vector", where the distance scale is derived from the parallax angle using the sine rule. Since the optimization variable is the angle rather than the depth scale itself, the observation error first acts on the angle variable, and then affects the scale and coordinates through mapping, making it easier for the variable increment in the normal equation and the observation error to remain within the same order of magnitude.

[0072] Meanwhile, this invention also incorporates the camera center into the parallax angle parameterization, so that the scale of the camera center and the three-dimensional connection point are derived from the geometric relationship of the same anchor point baseline. This avoids the scale inconsistency and unstable propagation that may be introduced when the camera center still participates in optimization in rectangular coordinates, thereby further reducing the system's sensitivity to small parallax observation errors.

[0073] Therefore, under conditions of small parallax and weak control, this invention can relatively reduce the amplification effect of "observation error - variable error" and improve the steady-state performance and reliability of adjustment solutions.

[0074] The above embodiments also include an analysis of the reduction of initial value dependence; specifically including: In traditional Cartesian coordinate parameter space, the objective function of bundle adjustment tends to exhibit a "flat valley" characteristic under small parallax conditions: the curvature in the depth direction is very small, making it difficult to balance the plane and depth directions in the iterative update step size, resulting in slow convergence speed and strong dependence on the initial value.

[0075] In this invention, the camera center position, the three-dimensional connection point position, and the camera pose are all uniformly expressed in the angle domain, the unknown quantities have the same dimensions, and the parallax angle is directly used as an optimization variable to participate in the iteration. Since the angle variable still maintains an observable scale of change under small parallax conditions, the curvature distribution of the objective function in the angle parameter space is relatively more balanced, thereby reducing the adverse effects of the "approximately flat depth direction" of the rectangular coordinates.

[0076] To further illustrate the robustness of the angle parameter space to initial values, depth-related unknowns in the Cartesian coordinate system (such as equivalent distance scales) can be considered. ) and parallax angle oh The mapping is considered a reparameterization: according to the sine theorem, when oh When smaller, right oh The derivative contains The term makes the objective function have very small curvature in the depth direction, easily forming "valleys", thus making it sensitive to initial values; while with oh When variables of equal angle are used as unknowns, the optimization process is carried out directly within the bounded angular domain, the variable scale is more consistent, and iterative updates are more likely to obtain a stable descent direction and convergence.

[0077] Furthermore, this invention also parameterizes the parallax angle of the camera center, so that the camera center and the three-dimensional connection points are both constrained by the same anchor baseline and obtain the scale through the same geometric derivation. This reduces the coupling inconsistency caused by the mixed expression of "camera center rectangular coordinates - three-dimensional connection point angle variables", thereby making the numerical behavior of the overall objective function more stable.

[0078] Therefore, when there is a large range of initial value errors, the present invention can usually improve the convergence success rate of adjustment iteration and reduce the number of iterations, which is manifested as a reduced dependence on initial values.

[0079] It should be noted that the above "reduced dependence on initial value" is a summary description of convergence behavior from the perspective of variable expression and numerical properties, and does not limit the specific solver type; the Gauss-Newton or least squares iterative framework commonly used in this field can be used to implement the adjustment solution in step S4 of this invention.

[0080] Initial values ​​of angle parameters are obtained. To enable this invention to be implemented in existing production processes, initial values ​​of rectangular coordinates can first be obtained using conventional methods (e.g., coarse coordinates of the photography center provided by POS / GNSS, coarse coordinates of a three-dimensional point obtained by the relative orientation of adjacent images and the intersection with the foreground, or a coarse solution obtained by performing a round of traditional bundle adjustment), and then converted into the initial values ​​of angle parameters of this invention. Taking the photography center as an example, let its initial rectangular coordinates be... Then define , , The direction angle and elevation angle can be calculated by using the unit direction vectors corresponding to the direction angle and elevation angle.

[0081] Then by as well as The initial value of the parallax angle can be obtained by inversely calculating the distance scale formula from the center of the photograph to the main anchor point:

[0082] For three-dimensional connection points It can be constructed using the same analogy. , and Initial values ​​of camera attitude The values ​​can be provided by the POS / IMU or by the relative orientation results of the image. Through the above transformation, the initial values ​​of traditional rectangular coordinates can be mapped to the angle parameter space of this invention, thereby facilitating stable iterative solutions.

[0083] in The three-dimensional coordinates of the primary anchor point (prior control point) The primary and secondary anchor point baseline vectors, Baseline length The initial vector pointing from the main anchor point to the center of the image. This represents the initial scale value for the distance from the photography center to the main anchor point. It is a unit direction vector with components. , , , , It is by The initial values ​​of the direction angle and elevation angle obtained by reverse calculation The initial value of the parallax angle at the center of the photograph is obtained by inverse calculation using the distance scale.

[0084] Example 3 This embodiment provides a full-range bundle adjustment system with joint parallax angle parameterization of the camera and object point, including: Image point processing module: It is used to acquire remote sensing images of the target area, extract and match the observation values ​​of image points with the same name in all remote sensing images; Reference frame building module: It is used to build an angular reference frame based on the matched image point observations and the prior anchor points of two known spatial coordinates. Joint parameterization module: It is used to uniformly parameterize the camera attitude, the position of the photography center and the position of the three-dimensional connection point using the orientation angle, elevation angle and parallax angle in the angle reference frame; Adjustment solution module: It is used to construct collinear observation equations based on parameterized camera pose, image center position and 3D tie point position, and perform bundle adjustment joint iterative optimization to obtain optimized image center coordinates and 3D tie point coordinates.

[0085] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0086] It should be understood that the above description of the preferred embodiments is quite detailed, but this should not be construed as limiting the scope of protection of this invention. It is neither necessary nor possible to exhaustively describe all possible implementations. Those skilled in the art, guided by this invention, can make substitutions or modifications without departing from the scope of the claims, all of which fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A bundle adjustment method with joint parallax angle parameterization of camera and object point, characterized in that, Includes the following steps: Acquire remote sensing images of the target area, and extract and match the observation values ​​of image points with the same name in all remote sensing images; An angular reference frame is established based on the matched image point observations and the prior anchor points of two known spatial coordinates. In the aforementioned angle reference frame, the camera attitude, the position of the photography center, and the position of the three-dimensional connection point are uniformly parameterized using the orientation angle, elevation angle, and parallax angle. Based on the parameterized camera pose, image center position, and 3D tie point position, a collinear observation equation is constructed and solved by bundle adjustment and joint iterative optimization to obtain the optimized image center coordinates and 3D tie point coordinates.

2. The bundle adjustment method for joint parallax angle parameterization of camera and object point as described in claim 1, characterized in that, The two prior anchor points include a primary anchor point and a secondary anchor point. The angle reference frame is established with the primary anchor point as the origin and the baseline vector pointing from the primary anchor point to the secondary anchor point as the reference axis.

3. The bundle adjustment method for joint parallax angle parameterization of camera and object point as described in claim 2, characterized in that, The parameterization of the camera center position is determined in the following way: Based on the i The direction angle of the photography center With elevation angle Determine the first i The unit direction vector of each photography center is ; Based on the baseline formed by two prior anchor points, the first... i Unit direction vector of the camera center and the i Parallax angle of the photography center The geometric relationship of the triangle is used to derive the distance scale from the principal anchor point using the sine theorem. .

4. The bundle adjustment method for joint parallax angle parameterization of camera and object point as described in claim 3, characterized in that, No. i A photography center Distance scale to the main anchor point The calculation formula is: No. i A photography center The formula for calculating coordinates is: in, Baseline length; The coordinates of the primary anchor point; For the first i Parallax angle of a photography center; For the first i The angle between the observation direction of each photography center and the baseline direction.

5. The bundle adjustment method for joint parallax angle parameterization of camera and object point according to claim 2, characterized in that, The parameterization of the three-dimensional connection points is determined in the following way: Based on the j The orientation angle of each three-dimensional connection point With elevation angle Determine the first j The unit direction vector of each three-dimensional connection point is ; Based on the baseline formed by two prior anchor points, the first... j Unit direction vector of each three-dimensional connection point and the j parallax angle of three-dimensional connection points The geometric relationship of the triangle is used to derive the distance scale from the principal anchor point using the sine theorem. .

6. The bundle adjustment method for joint parallax angle parameterization of camera and object point as described in claim 5, characterized in that, No. j Three-dimensional connection points Distance scale to the main anchor point The calculation formula is: No. j Coordinates of the three-dimensional connection points for: in, Baseline length; The coordinates of the primary anchor point; for j Parallax angle of three-dimensional connection points; for j The angle between the observation direction of each three-dimensional connection point and the baseline direction.

7. The bundle adjustment method for joint parallax angle parameterization of camera and object point according to claim 1, characterized in that, Camera pose parameterization is determined in the following way: For the i The camera pose of the image is determined using the pose angle. Perform parameterization and generate the first... i pose rotation matrix of the image .

8. The bundle adjustment method for joint parallax angle parameterization of camera and object point according to claim 1, characterized in that, Constructing the collinear observation equations and performing bundle adjustment includes: For the i The first image j Given a point with the same name, its observation coordinates are... Through camera intrinsic parameter matrix Normalization yields the unit view direction vector in the camera coordinate system. ; The first j Coordinates of the three-dimensional connection points With the i Coordinates of the photography center The coordinate difference vector in the anchor point coordinate system, through the first... i pose rotation matrix of the image Transform to camera coordinate system to obtain object direction vector ; Construct collinear residual vectors ; The objective function is to perform nonlinear optimization to solve the problem by minimizing the sum of squares of collinear residuals of all image points with the same name.

9. The bundle adjustment method for joint parallax angle parameterization of camera and object point according to claim 1, characterized in that, Before the iterative optimization solution, a parameter initialization step is also included: Based on the initial estimated coordinates of the camera center and the estimated coordinates of the three-dimensional points in space, and combined with the prior anchor point coordinates and baseline vector, the initial values ​​of the orientation angle, elevation angle and parallax angle corresponding to each camera center and each three-dimensional point in space are calculated in reverse. The initial values ​​of the camera attitude angles are provided by external measurement data or image relative orientation results.

10. A bundle adjustment system with joint parallax angle parameterization of camera and object point, characterized in that, include: Image point processing module: It is used to acquire remote sensing images of the target area, extract and match the observation values ​​of image points with the same name in all remote sensing images; Reference frame building module: It is used to build an angular reference frame based on the matched image point observations and the prior anchor points of two known spatial coordinates. Joint parameterization module: It is used to uniformly parameterize the camera attitude, the position of the photography center and the position of the three-dimensional connection point using the orientation angle, elevation angle and parallax angle in the angle reference frame; Adjustment and solution module: It is used to construct collinear observation equations based on parameterized camera pose, camera center position and 3D tie point position, and perform bundle adjustment joint iterative optimization to obtain optimized camera center coordinates and 3D tie point coordinates. The bundle adjustment system with joint parallax angle parameterization of the camera and object points is used to perform the steps in the bundle adjustment method with joint parallax angle parameterization of the camera and object points as described in any one of claims 1-9.

Images