Low-slow small-target large-field-of-view photogrammetry system external parameter calibration method based on linear extrapolation

Abstract

The invention relates to a linear extrapolation-based external parameter calibration method for a low-slow small-target large-field-of-view photogrammetry system, which comprises the following steps of: based on a three-dimensional point control field of a large-field-of-view photogrammetry system, selecting point pairs to construct straight lines meeting conditions, and forming a spatial linear control field by the constructed straight lines; calculating an external parameter initial value by adopting a direct linear solution based on a space straight line based on the space straight line control field; an error model is constructed by taking an external parameter initial value as an initial value of iterative optimization and taking a re-projection error of a minimized straight line as a target function, so that the deviation between the re-projection straight line and an observation straight line is minimum, nonlinear iterative optimization is performed on parameters, accurate calibration of the external parameters of the camera is completed, and accurate measurement of a low-speed small target is realized. Compared with the prior art, the method has the advantages that accurate, reliable and low-cost calibration of the external parameters of the low-slow small-target photogrammetry system under the condition of a large field of view can be realized, and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of calibration of photogrammetry systems, and more particularly to an external parameter calibration method for a large field of view photogrammetry system for low, slow and small targets based on straight line extrapolation. Background Art

[0002] In view of the security threats posed by various low-altitude, slow-speed and small (low, slow and small) flying vehicles such as "unauthorized flight" and "random flight" to military / civil facilities, when using an optical detection method based on a photogrammetry system to obtain the three-dimensional trajectory of a target, it is necessary to calibrate the photogrammetry system. Traditional calibration methods often rely on cooperative targets to uniformly establish a point control field within the camera's field of view to obtain the external parameters of the camera. For the measurement of low, slow and small targets, the shooting scene often includes a large part of the sky background and has a large field of view, resulting in difficulty for cooperative targets to uniformly and completely cover the measurement field of view. Moreover, the large field of view calibration method based on laser beams has defects such as low daytime visibility, potential safety hazards, and high cost of laser devices. Therefore, existing methods are difficult to achieve accurate, reliable and low-cost calibration of the measurement system. Summary of the Invention

[0003] The purpose of the present invention is to provide an external parameter calibration method for a large field of view photogrammetry system for low, slow and small targets based on straight line extrapolation. Cooperative targets for calibration are arranged in the local space near the camera, and a spatial straight line control field for the complete measurement field of view is constructed using a finite number of control points in the local space. For the spatial straight line, the direct linear solution method is used to obtain the initial value of the external parameters of the photogrammetry system. Subsequently, an error model is established with the minimization of the spatial straight line reprojection error as the objective function, and the external parameters are further refined through non-linear iterative optimization, thereby achieving accurate calibration of the external parameters of the photogrammetry system for low, slow and small targets under large field of view conditions.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] An external parameter calibration method for a large field of view photogrammetry system for low, slow and small targets based on straight line extrapolation, comprising the following steps:

[0006] S1, based on the three-dimensional point control field of the large field of view photogrammetry system, select point pairs to construct straight lines that meet the conditions, and form a spatial straight line control field from the constructed straight lines;

[0007] S2, based on the spatial straight line control field, calculate the initial value of the external parameters using the direct linear solution method based on spatial straight lines;

[0008] S3. Use the initial value of the external parameters as the initial value for iterative optimization. Construct an error model with minimizing the reprojection error of the straight line as the objective function, so that the deviation between the reprojection straight line and the observed straight line is minimized, and perform non-linear iterative optimization on the parameters to complete the accurate calibration of the camera's external parameters, thereby achieving the precise measurement of low, slow, and small targets.

[0009] In the point control field pre-laid in the local space near the camera, select three-dimensional point pairs to construct space straight lines. The constructed straight lines satisfy the following conditions:

[0010] The total number is not less than 6;

[0011] The straight lines are not in the same plane;

[0012] The straight lines are evenly distributed in all directions in space, that is, the vectors forming the straight lines have uniform angular directions in space.

[0013] The S2 includes the following steps:

[0014] S21. Determine the plane equation satisfied by the points on the intersection line of two planes and perform simplification processing. Establish a collinearity condition equation based on the correspondence between the points on the image and the points in the object space, and obtain the expression of a straight line based on the simplified plane equation and the collinearity condition equation;

[0015] S22. Construct a system of linear equations based on the expressions of multiple straight lines, and solve the coefficients in the system of linear equations according to the straight line data in the space straight line control field;

[0016] S23. Calculate the camera projection matrix according to the coefficients of the obtained system of equations, and combine the camera internal parameter matrix obtained by prior calibration to calculate the rotation matrix and translation vector.

[0017] The S21 is specifically as follows:

[0018] Use two intersecting planes to represent a straight line. The points on the intersection line of the two planes must satisfy the two plane equations:

[0019]

[0020] Among them, (x, y, z) are the coordinates in three-dimensional space, and A, B, C, D are the coefficients of the plane equation;

[0021] Assume that the straight line does not pass through the origin, then D≠0. Divide both sides of the plane equation by D to make the equation non-homogeneous:

[0022]

[0023] And simplify it to:

[0024]

[0025] Among them,

[0026] Express the above formula in the form of a function:

[0027]

[0028] And further simplify it to:

[0029]

[0030] Among them,

[0031] According to the collinearity condition equation that describes the corresponding relationship between points on the image and points in the object space:

[0032]

[0033] Among them, m 0 ~m 11 Are the coefficients in the camera projection matrix M;

[0034] The expression of the image straight line l is u = av + b, where u and v are the two-dimensional coordinates of points on the image. Substitute it into the collinearity condition equation together with the function form of the simplified plane equation to obtain:

[0035]

[0036] Divide both sides by m 11 , and at the same time let s i = m i / m 11 , to obtain the expression of a straight line:

[0037]

[0038] The expression of the straight line is a non-homogeneous linear equation system about the coefficients s 0 ~s 10 .

[0039] The straight line equation system is expressed as:

[0040]

[0041] The left side of the equation is composed of image straight line parameters and space straight line parameters. Solve the straight line equation system according to the data of 6 space straight lines to obtain the coefficients s 0 ~s 10 .

[0042] The specific S23 is:

[0043] Set the parameter matrix of the camera, i.e., the rotation matrix R, the translation vector t, and the intrinsic matrix K:

[0044]

[0045] Among them, r 0 ~r 8 are the coefficients of the rotation matrix, representing the directions of the x, y, and z axes of the camera coordinate system in the world coordinate system; t x , t y , t z are the translation parameters from the origin of the world coordinate system to the origin of the camera coordinate system; f x , f y are the focal lengths of the camera on the x and y axes of the image; c x , c y are the coordinates of the principal point of the image on the x and y axes of the image;

[0046] Obtain the coefficients of each item in the camera projection matrix M according to the camera projection expression:

[0047]

[0048] According to the properties of the rotation matrix, we have Substitute the obtained s 0 ~s 10 and we get:

[0049]

[0050] From m i = s i m 11 , we obtain all the remaining elements in the camera projection matrix, namely m 0 ~m 10 ;

[0051] According to the camera intrinsic matrix obtained by prior calibration, calculate the coefficients of each item in the rotation matrix and the translation vector through the coefficients of each item in the projection matrix M to complete the initial solution of the camera extrinsic parameters:

[0052]

[0053] t z = m 11 , r 6 = m 8 , r 7 = m 9 , r 8 = m 10 .

[0054] The S3 includes the following steps:

[0055] S31, Spatial line parameterization;

[0056] S32, Construct a spatial line observation error model;

[0057] S33, Use the initial value of the external parameters as the initial value of iterative optimization, and perform non - linear parameter optimization based on the error model.

[0058] The specific content of S31 is as follows: Introduce two representation methods of a line for line parameterization: the Plücker coordinate representation of the line and the orthogonal representation of the line;

[0059] Assume that the homogeneous coordinates of two spatial points are X 1 =(x 1 , y 1 , z 1 , r 1 ) T and X 2 =(x 2 , y 2 , z 2 , r 2 ), and their corresponding non - homogeneous coordinates are respectively The Plücker coordinates of a line are determined by two spatial points, and the Plücker coordinates are defined as:

[0060]

[0061] where n is the normal vector of the plane formed by the line and the origin of the current coordinate system, and v is the direction vector of the line;

[0062] The line in the world coordinate system and the line in the camera coordinate system The transformation relationship is described in Plücker coordinates as:

[0063]

[0064] where R cw and t cw are respectively the rotation and translation parameters from the world coordinate system to the camera coordinate system, n w , n c are respectively the normal vectors of the planes formed by the lines in the world coordinate system and the camera coordinate system and the corresponding coordinate system origins, and v w , v c are respectively the direction vectors of the lines in the world coordinate system and the camera coordinate system; The transformation from the spatial line in the camera coordinate system to the line on the camera imaging plane is expressed as:

[0065]

[0066] where l cis a straight line on the imaging plane, f x , f y are the focal lengths of the camera on the x and y axes of the image;

[0067] By performing QR decomposition on the Plücker coordinates, an orthogonal representation of the straight line is obtained quickly, serving as the basis for parameter optimization: Based on the orthogonal constraint of the Plücker coordinates, its transformation relationship is directly obtained, corresponding to the Lie algebra W under the special orthogonal group U and the rotation angle Θ:

[0068]

[0069] Let cos(Θ) = w 1 , sin(Θ) = w 2 , and the transformation relationship between the orthogonal representation of the straight line and the Plücker coordinates is obtained:

[0070]

[0071] The specific S32 is as follows: Use the method of 3D-2D projection of the space straight line, and define the error between the observed straight line on the image and the reprojection straight line based on the endpoint distance of the projected straight line, denoted as the image-side reprojection error;

[0072] For the two endpoints x s = [u 1 , v 1 , 1] and x e = [u 2 , v 2 , 1] of the straight line segment l on the image, where u 1 , v 1 and u 2 , v 2 are the two-dimensional coordinates corresponding to the endpoints x s and x e respectively. The perpendicular distances e s and e e from the two endpoints to the reprojection straight line l' are obtained based on the following geometric principle;

[0073] In the homogeneous coordinate system, the expression of the reprojection straight line l' on the imaging plane is written as l' = [l 1 l 2 l 3 T , in the form of the general equation au + bv + c = 0 of a straight line in the 2D plane, that is, l = [a b c] T , where u and v are the coordinates of the points on the straight line, and [l 1 l 2 = [a b] is the normal vector parameter of the straight line, and l 3 = c is the intercept parameter of the straight line; ​

[0074] Substitute the endpoints \(x\) s and \(x\) e into the expression of the reprojection line \(l'\) on the imaging plane to obtain: Two intermediate quantities. Divide the two intermediate quantities by the length of the normal vector of the line for normalization to obtain the perpendicular distances \(e\) s and \(e\) e from the two endpoints to the reprojection line \(l'\):

[0075]

[0076] Based on the perpendicular distances \(e\) s and \(e\) e from the two endpoints to the reprojection line \(l'\), obtain the image - side reprojection error. The image - side reprojection error is a geometric non - linear error, and its definition is as follows:

[0077]

[0078] Specifically, \(S23\) is:

[0079] Assume that the sequence of corresponding spatial lines and image - side lines is \(\{(l 1 ,L 1 ),(l 2 ,L 2 ),...,(l N ,L N )\}, where \(N\) is the number of paired lines, \(L\) and \(l\) are the spatial line and its corresponding image - side line. Then, the camera extrinsic parameter optimization problem corresponding to the spatial line and the image - side line is expressed by the following formula:

[0080]

[0081] where \(d(l i ,L i )\) represents the distance between the image - side line and the reprojection spatial line. The least - squares solution of the camera extrinsic parameters is obtained by minimizing the error function \(E\);

[0082] By solving the Jacobian matrix of the image - side reprojection error \(e l with respect to the camera extrinsic parameter increment \(\delta ζ and the increment \(\delta θ of the orthogonal representation of the line, the solution of the line reprojection error is realized; two Jacobian matrices are obtained through the chain - rule of differentiation:

[0083]

[0084] where,

[0085]

[0086] Among them

[0087]

[0088] The Jacobian matrix between the image-side reprojection error and the state variables is obtained through the above formula, and the Levenberg-Marquardt nonlinear optimization calculation method is used to achieve the accurate solution of the external parameters of the camera.

[0089] Compared with the prior art, the present invention has the following beneficial effects:

[0090] Aiming at the problem that it is difficult to accurately, reliably and low-cost calibrate the external parameters of a low, slow and small target photogrammetry system under large field-of-view shooting conditions, the present invention proposes a calibration method based on straight-line extrapolation. By constructing a spatial straight-line control field containing more than 6 straight lines evenly distributed in all directions in the local space near the camera, and using the direct linear solution method for the spatial straight lines to obtain the initial values of the external parameters of the photogrammetry system. Subsequently, an error model is established with minimizing the spatial straight-line reprojection error as the objective function, and the external parameters are further refined through nonlinear iterative optimization, realizing the accurate calibration of the camera external parameters without introducing new equipment or devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 is the flowchart of the method of the present invention;

[0092] Figure 2 is the schematic diagram for constructing the spatial straight-line control field based on the point control field;

[0093] Figure 3 is the schematic diagram of the image-side reprojection error based on the endpoint distance;

[0094] Figure 4 is the schematic diagram of the three-dimensional trajectory measurement test scene of low, slow and small targets within 120m viewing distance in an embodiment, where (a) is the top view; (b) is the side view;

[0095] Figure 5 is the sample diagram for arranging the test check points at the shooting distance of 120m in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0096] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and gives the detailed implementation manners and specific operation processes, but the protection scope of the present invention is not limited to the following embodiments.

[0097] This embodiment provides a method for calibrating the external parameters of a low, slow and small target large field-of-view photogrammetry system based on straight-line extrapolation, which is divided into three parts: constructing a spatial straight-line control field, calculating the initial values of the external parameters, and backend optimization, asFigure 1 As shown in the figure, it includes the following steps:

[0098] S1. Based on the three-dimensional point control field of the large field-of-view photogrammetry system, select point pairs to construct straight lines that meet the conditions, and form a spatial straight line control field with the constructed straight lines.

[0099] In this step, based on the traditional point control field, a method for constructing a spatial straight line control field is proposed. By selecting point pairs in the point control field to form straight lines, the construction of the spatial straight line control field is completed. The specific implementation process is as follows:

[0100] As Figure 2 shown in the figure, in the point control field pre-laid in the local space near the camera, select three-dimensional point pairs to construct spatial straight lines (such as Figure 2 L 1 ~L n in the figure), and the constructed straight lines meet the following conditions:

[0101] ① The total number is not less than 6;

[0102] ② The straight lines are not in the same plane;

[0103] ③ The straight lines are evenly distributed in all directions in space, that is, the vectors forming the straight lines have uniform angular directions in space.

[0104] These straight lines formed by three-dimensional point pairs and meeting the conditions constitute the spatial straight line control field.

[0105] S2. Based on the spatial straight line control field, use the direct linear solution method based on spatial straight lines to calculate the initial value of the external parameters.

[0106] In this step, based on the traditional direct linear transformation (DLT) method based on three-dimensional points, a DLT solution method based on spatial straight lines is derived. By combining the expression of the spatial straight line with the collinearity condition equation, the functional relationship between the spatial straight line and the image-side straight line is obtained. The specific implementation process is as follows:

[0107] S21. Determine the plane equation satisfied by the points on the intersection line of two planes, and perform simplification processing. Establish a collinearity condition equation according to the correspondence relationship between the points on the image and the points in the object space, and obtain the expression of a straight line based on the simplified plane equation and the collinearity condition equation.

[0108] A straight line can be represented by two intersecting planes, and this representation method of a straight line is called the general representation method of a straight line. The points on the intersection line of two planes must satisfy the two plane equations, as shown in Equation (1):

[0109] Represent a straight line with two intersecting planes, and the points on the intersection line of the two planes must satisfy the two plane equations:

[0110]

[0111] Among them, (x, y, z) are the coordinates in three-dimensional space, and A, B, C, D are the coefficients of the plane equation.

[0112] Assume that the straight line does not pass through the origin, then D≠0. Divide both sides of the plane equation by D to make the equation non-homogeneous:

[0113]

[0114] And simplify it to:

[0115]

[0116] Among them,

[0117] Express the above formula in the form of a function:

[0118]

[0119] And further simplify it to:

[0120]

[0121] Among them,

[0122] According to the collinearity condition equation that describes the corresponding relationship between points on the image and points in the object space:

[0123]

[0124] Among them, m 0 ~m 11 are the coefficients in the camera projection matrix M.

[0125] The expression of the image straight line l is u = av + b. Among them, u, v are the two-dimensional coordinates of the points on the image. Substitute it into the collinearity condition equation (6) together with the simplified plane equation function form (5) to get:

[0126]

[0127] Divide both sides by m 11 , and at the same time let s i = m i / m 11 , to obtain the expression of a straight line:

[0128]

[0129] Equation (8) is the expression of a straight line, which is a function about the coefficients s 0 ~s10 Non - homogeneous linear equations

[0130] S22. Construct a system of linear equations based on the expressions of multiple lines, and solve for the coefficients in the system of linear equations according to the line data in the spatial line control field

[0131] Based on Equation (8), under the condition of having multiple lines, a system of equations as shown in Equation (9) can be formed

[0132]

[0133] This system of equations has a total of 11 unknowns, and each spatial line provides two conditional equations. Therefore, at least 6 spatial line data are required to solve this system of equations. The left side of the equation consists of image line parameters and spatial line parameters, and the coefficients s 0 ~s 10 can be solved according to Equation (9).

[0134] S23. Calculate the camera projection matrix according to the coefficients of the obtained system of equations, and calculate the rotation matrix and translation vector in combination with the camera internal parameter matrix obtained by prior calibration

[0135] Set the parameter matrix of the camera, namely the rotation matrix R, the translation vector t, and the internal parameter matrix K

[0136]

[0137] Among them, r 0 ~r 8 are the coefficients of the rotation matrix, representing the directions of the x, y, and z axes of the camera coordinate system in the world coordinate system; t x , t y , t z are the translation parameters from the origin of the world coordinate system to the origin of the camera coordinate system; f x , f y are the focal lengths of the camera on the x and y axes of the image; c x , c y are the coordinates of the principal point of the image on the x and y axes of the image

[0138] Obtain the coefficients in the camera projection matrix M according to the camera projection expression

[0139]

[0140] According to the properties of the rotation matrix, we have Substitute the obtained s 0 ~s 10 and we get

[0141]

[0142] From m i = s i m 11 , all the remaining elements in the camera projection matrix, i.e., m 0 ~m 10 .

[0143] According to the camera internal parameter matrix obtained by prior calibration, the coefficients of the rotation matrix and the translation vector can be calculated through the coefficients in the projection matrix M, and the initial solution of the camera external parameters is completed, as shown in Equation (14):

[0144]

[0145] t z = m 11 , r 6 = m 8 , r 7 = m 9 , r 8 = m 10 (14)

[0146] S3. Take the initial value of the external parameters as the initial value of iterative optimization, construct an error model with minimizing the reprojection error of the straight line as the objective function, make the deviation between the reprojection straight line and the observed straight line the smallest, and perform non-linear iterative optimization on the parameters to complete the accurate calibration of the camera external parameters, so as to achieve the precise measurement of low, slow and small targets.

[0147] The specific implementation process of this step is as follows:

[0148] S31. Parametrize the spatial straight line.

[0149] In this embodiment, two representation methods of the straight line are introduced for straight line parameterization: the Plücker coordinate representation of the straight line and the orthogonal representation of the straight line.

[0150] Since a spatial straight line can be determined by two points, it is assumed that the homogeneous coordinates of two spatial points are X 1 = (x 1 , y 1 , z 1 , r 1 ) T and X 2 = (x 2 , y 2 , z 2 , r 2 ), and their corresponding inhomogeneous coordinates are respectively The Plücker coordinates of the straight line are determined by two spatial points, and the Plücker coordinates are defined as:

[0151]

[0152] Among them, n is the normal vector of the plane formed by the straight line and the origin of the current coordinate system, and v is the direction vector of the straight line.

[0153] The Plücker matrix can not only conveniently represent the coordinate transformation and projection transformation of spatial straight lines, but also has no limitations and can represent any straight line in space. However, its over-parameterization and orthogonal constraint characteristics make it inapplicable to iterative optimization calculations. Therefore, this method selects the orthogonal representation of the straight line as the basis for optimization, which only requires 4 parameters to represent a three-dimensional space straight line and can be quickly obtained by performing QR decomposition on the Plücker coordinates.

[0154] Straight line in the world coordinate system and the straight line in the camera coordinate system The transformation relationship is described in Plücker coordinates as:

[0155]

[0156] Among them, R cw and t cw are the rotation and translation parameters from the world coordinate system to the camera coordinate system respectively. n w , n c are the normal vectors of the planes formed by the straight lines in the world coordinate system and the camera coordinate system and the corresponding coordinate system origins respectively. v w , v c are the direction vectors of the straight lines in the world coordinate system and the camera coordinate system respectively. The transformation between the spatial straight line in the camera coordinate system and the straight line on the camera imaging plane is expressed as:

[0157]

[0158] Among them, l c is the straight line on the imaging plane, f x , f y are the focal lengths of the camera on the x and y axes of the image.

[0159] The orthogonal representation of the straight line is quickly obtained by performing QR decomposition on the Plücker coordinates as the basis for parameter optimization: Based on the orthogonal constraint of the Plücker coordinates, its transformation relationship is directly obtained, corresponding to the Lie algebra W under the special orthogonal group U and the rotation angle Θ condition:

[0160]

[0161] Let cos(Θ) = w 1 , sin(Θ) = w 2 , and the transformation relationship between the orthogonal representation of the straight line and the Plücker coordinates is obtained:

[0162]

[0163] S32. Construct a spatial straight line observation error model.

[0164] Use the 3D-2D projection of the spatial straight line, and define the error between the observed straight line and the reprojected straight line on the image based on the endpoint distance of the straight line after projection, denoted as the image-side reprojection error. As shown in Figure 3 , where the bold solid line is the reprojected straight line and the other solid line is the observed straight line.

[0165] For the two endpoints x s = [u 1 , v 1 , 1] and x e = [u 2 , v 2 , 1] of the straight line segment l on the image, where u 1 , v 1 , u 2 , v 2 are the two-dimensional coordinates corresponding to the endpoints x s , x e respectively. The perpendicular distances e s , e e from the two endpoints to the reprojected straight line l' are obtained based on the following geometric principle:

[0166] In the homogeneous coordinate system, the expression of the reprojected straight line l' on the imaging plane is written as l' = [l 1 l 2 l 3 T , in the form of the general equation au + bv + c = 0 of a straight line in the 2D plane, that is, l = [a b c] T , where u and v are the coordinates of the points on the straight line, [l 1 l 2 = [a b] is the normal vector parameter of the straight line, and l 3 = c is the intercept parameter of the straight line;

[0167] Substitute the endpoints x s , x e into the expression of the reprojected straight line l' on the imaging plane to obtain: Two intermediate quantities. These two values are not the perpendicular distances from the two endpoints to the reprojected straight line and need to be further normalized. Divide the two intermediate quantities by the length of the normal vector of the straight line for normalization to obtain the perpendicular distances e s , e e from the two endpoints to the reprojected straight line l':

[0168]

[0169] ​Based on the perpendicular distances e s and e e from two endpoints to the reprojection line l′, the image-side reprojection error is obtained. The image-side reprojection error is a geometric non-linear error and is defined as follows:

[0170]

[0171] S33. Take the initial value of the external parameters as the initial value of the iterative optimization, and perform non-linear parameter optimization based on the error model.

[0172] Assume that the sequence of corresponding spatial lines and image-side lines is {(l 1 , L 1 ), (l 2 , L 2 ),..., (l N , L N )}, where N is the number of paired lines, L and l are the spatial line and its corresponding image-side line. Then, the camera external parameter optimization problem corresponding to the spatial line and the image-side line is expressed by the following formula:

[0173]

[0174] where d(l i , L i ) represents the distance between the image-side line and the reprojection spatial line. The least-squares solution of the camera external parameters is obtained by minimizing the error function E.

[0175] By solving the Jacobian matrix of the image-side reprojection error e l with respect to the camera external parameter increment δ ζ and the increment δ θ of the orthogonal representation of the line, the solution of the line reprojection error is realized; two Jacobian matrices are obtained through the chain rule of differentiation:

[0176]

[0177] where,

[0178]

[0179] where

[0180]

[0181] The Jacobian matrix between the image-side reprojection error and the state variables is obtained through the above formula, and the Levenberg-Marquardt non-linear optimization calculation method is used to realize the accurate solution of the camera external parameters.

[0182] To verify the effectiveness and reliability of the method proposed in the present invention, a three-dimensional trajectory measurement experiment of low, slow and small targets within a visual range of 120 m was carried out. The calibration accuracy of the photogrammetry system was analyzed through the check points arranged in the visual field, and a comparison was made with the traditional calibration method based on the point control field.

[0183] (1) Test scenario:

[0184] As Figure 4 shown, two cameras were arranged at a position about 120 m horizontally from the far end of the measurement area. The baseline length between the two cameras was 20 m. The cameras both used 50 mm fixed-focus lenses, and the imaging horizontal field of view angle was about 20°. The two cameras were arranged at a certain intersection angle to cover the target airspace. The size of the far-end visual field of the area was about 43 m × 34 m.

[0185] As Figure 5 shown, a movable metal frame was used to fix the circular artificial marking points and move them multiple times within the shooting visual field. Each time it moved, the total station was used to measure the three-dimensional coordinates of the center of the marking points, and at the same time, the two arranged cameras were used to capture them to complete the layout and measurement of the check points.

[0186] In addition, as Figure 4 shown, the arranged control field area was smaller than the detection area size, only covering the adjacent area of about 35 m with a horizontal distance of 55 - 90 m from the camera, and the height was about 6 m, while the check points used to verify the accuracy were arranged within the entire measurement area with a distance of 35 - 120 m from the camera.

[0187] (2) Test results:

[0188] After calculating and obtaining the external parameters of the camera respectively through the calibration method based on the traditional point control field and the calibration method based on the space straight line proposed in the present invention, the three-dimensional reconstruction of the check points was carried out. Taking the total station measurement result as the reference and the root mean square error (RMSE) as the index, the error level of the calibration result was evaluated, as shown in Table 1.

[0189] Table 1 Positioning error of check points in the 120 m shooting distance test of the camera external parameter calibration method based on the traditional point control field

[0190]

[0191]

[0192] As shown in Table 1, for the 120 m visual range verification experiment, the root mean square errors of the photogrammetry results of the check points and the total station measurement coordinates obtained by the camera external parameter calibration method based on the traditional point control field were 0.34 cm, 3.30 cm, and 0.42 cm respectively in the three directions, and the root mean square error value in the synthetic direction was 3.34 cm.

[0193] Table 2 Positioning error of test check points for the camera external parameter calibration method based on spatial straight lines at a shooting distance of 120m

[0194]

[0195]

[0196] As shown in Table 2, for the 120m visual distance verification test, the root mean square errors of the photogrammetric positioning results of the artificial target obtained by the camera external parameter calibration method based on spatial straight lines and the total station measurement coordinates are 0.34cm, 2.14cm, and 0.28cm in the three directions respectively, and the root mean square error value in the synthetic direction is 2.18cm.

[0197] In summary, on the premise of only using the point control field in the local space of the adjacent camera, the calibration accuracy of the method of this patent is better than that of the traditional calibration method based on the point control field in the overall measurement area of a large range. The accuracy improvement in the synthetic direction can reach about 34%, which reflects the reliability of this method and the effectiveness of the "extrapolation" characteristic under the condition of limited layout of the control field.

[0198] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of this application based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation, characterized in that: The following steps are involved: S1, based on the three-dimensional point control field of the large-field photogrammetry system, point pairs are selected to construct straight lines that meet the conditions, and the constructed straight lines constitute the spatial straight line control field; S2, based on the space straight line control field, the initial value of the external parameter is calculated using a direct linear solution based on the space straight line; S3, taking the initial value of the extrinsic parameter as the initial value of the iterative optimization, constructing an error model with minimizing the reprojection error of the straight line as the objective function, so that the deviation between the reprojected straight line and the observed straight line is minimized, and the parameters are nonlinearly iteratively optimized to complete the accurate calibration of the camera extrinsic parameters, thereby achieving accurate measurement of low, slow and small targets.

2. According to the method of extrinsic calibration of a low, slow, small target and large field of view photogrammetry system based on linear extrapolation in claim 1, it is characterized in that: In the point control field pre-arranged in the local space near the camera, select a 3D point pair to construct a spatial straight line. The constructed straight line meets the following conditions: The total number is not less than 6; The lines are not in the same plane; Straight lines are evenly distributed in all directions in space, that is, the angles of the vectors that make up the straight lines in space are evenly distributed.

3. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 1 is characterized in that: The S2 comprises the following steps: S21, determining the plane equations satisfied by the points on the intersection of the two planes, and simplifying them, establishing a collinearity condition equation based on the corresponding relationship between the points on the image and the points in the object space, and obtaining an expression of a straight line based on the simplified plane equation and the collinearity condition equation; S22, constructing a linear equation group based on the expressions of the plurality of straight lines, and solving coefficients in the linear equation group according to the straight line data in the spatial straight line control field; S23, calculating the camera projection matrix according to the coefficients of the obtained equation group, and calculating the rotation matrix and the translation vector in combination with the camera intrinsic parameter matrix obtained by pre-calibration.

4. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 3 is characterized in that: The S21 is specifically: Use two intersecting planes to represent a straight line. The points on the intersection of the two planes must satisfy two plane equations: Among them, (x, y, z) are coordinates in three-dimensional space, A, B, C, D are the coefficients of the plane equation; Assuming that the straight line does not pass through the origin, then D≠0, divide both sides of the plane equation by D to make the equation non-homogeneous: And simplify it to: in, Express the above formula as a function: And further simplified to: in, According to the collinearity condition equation describing the correspondence between the points on the image and the midpoints in the object space: Among them, m0~m 11 are the coefficients in the camera projection matrix M; The expression of the image line l is u=av+b, where u and v are the two-dimensional coordinates of the points on the image. Substituting it and the simplified plane equation function form into the collinearity condition equation, we get: Divide both sides by m 11 , and let s i =m i / m 11 , we get an expression for a straight line: The expression of the straight line is a relation between coefficients s0~s 10 Non-homogeneous linear system of equations.

5. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 4 is characterized in that: The linear equations are expressed as: The left side of the equation is composed of image line parameters and space line parameters. The linear equations are solved according to the data of the six space lines to obtain coefficients s0~s 10 .

6. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 5, characterized in that: The S23 is specifically: Set the camera's parameter matrix, namely the rotation matrix R, translation vector t, and intrinsic parameter matrix K: Among them, r0~r8 are the coefficients of the rotation matrix, which represent the directions of the x, y, and z axes of the camera coordinate system in the world coordinate system; t x ,t y ,t z is the translation parameter from the origin of the world coordinate system to the origin of the camera coordinate system; f x ,f y is the focal length of the camera on the x and y axes of the image; c x ,c y are the coordinates of the principal point on the x and y axes of the image; According to the camera projection expression, the coefficients of the camera projection matrix M are obtained: According to the properties of the rotation matrix, The solved s0~s 10 Substituting in: By m i =s i m 11 , get all the remaining elements in the camera projection matrix, namely m0~m 10 ; According to the camera intrinsic parameter matrix obtained in advance, the coefficients in the rotation matrix and translation vector are calculated through the coefficients in the projection matrix M to complete the initial solution of the camera extrinsic parameters: t z =m 11 ,r6=m8,r7=m9,r8=m 10 。 7. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 1, characterized in that: The S3 comprises the following steps: S31, space line parameterization; S32, constructing a spatial straight line observation error model; S33, taking the initial value of the external parameter as the initial value of the iterative optimization, and performing nonlinear optimization of the parameter based on the error model.

8. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 7, characterized in that: The S31 specifically includes: introducing two methods of representing a straight line to parameterize the straight line: a Plücker coordinate representation of a straight line and an orthogonal representation of a straight line; Assume that the homogeneous coordinates of two space points are X1=(x1,y1,z1,r1) T and X2=(x2,y2,z2,r2), the corresponding non-homogeneous coordinates are The Plücker coordinates of a line are determined by two points in space. Defined as: Where n is the normal vector of the plane formed by the straight line and the origin of the current coordinate system, and v is the direction vector of the straight line; Straight line in world coordinate system The straight line in the camera coordinate system The transformation relationship of is described in Plücker coordinates as: Among them, R cw and t cw are the rotation and translation parameters from the world coordinate system to the camera coordinate system, n w 、n c are the normal vectors of the plane formed by the straight line in the world coordinate system and the camera coordinate system and the origin of the corresponding coordinate system, v w 、v c are the direction vectors of the straight line in the world coordinate system and the camera coordinate system respectively; the transformation from the spatial straight line in the camera coordinate system to the straight line on the camera imaging plane is expressed as: Among them, l c is a straight line on the imaging plane, f x ,f y is the focal length of the camera on the x and y axes of the image; By performing QR decomposition on the Plücker coordinates, the orthogonal representation of the straight line is quickly obtained as the basis for parameter optimization: Based on the orthogonal constraints of the Plücker coordinates, the transformation relationship is directly obtained, corresponding to the special orthogonal group U and the Lie algebra W under the rotation angle Θ: Let cos(Θ)=w1, sin(Θ)=w2, and we can get the conversion relationship between the orthogonal representation of the straight line and the Plücker coordinates:

9. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 8, characterized in that: The S32 specifically includes: using a 3D-2D projection method of a space straight line, and defining an error between the observed straight line on the image and the reprojected straight line based on the distance between the endpoints of the projected straight line, which is recorded as an image-side reprojection error; For the two endpoints x of the straight line segment l on the image s =[u1,v1,1] and x e =[u2,v2,1], where u1,v1,u2,v2 are the endpoints x s 、x e The corresponding two-dimensional coordinates, the perpendicular distance e from the two endpoints to the reprojected line l′ d 、e e It is based on the following geometric principles: In the homogeneous coordinate system, the expression of the reprojected line l' on the imaging plane is written as l' = [l1 l2 l3] T , which is like the general equation of a straight line in a 2D plane, au+bv+c=0, i.e. l=[abc] T , where u and v are the coordinates of the point on the line, [l1l2] = [ab] is the normal vector parameter of the line, and l3 = c is the intercept parameter of the line; Set the endpoint x s 、x e Substituting the expression of the reprojected line l′ on the imaging plane, we get: Two intermediate quantities, divide the two intermediate quantities by the length of the normal vector of the line Normalize and get the vertical distance e from the two endpoints to the reprojection line l′ s 、e e : Based on the perpendicular distance e between the two endpoints and the reprojected line l' s 、e e The image-side reprojection error is obtained. The image-side reprojection error is a geometric nonlinear error, which is defined as follows:

10. The method for calibrating external parameters of a low, slow, small target, large field of view photogrammetry system based on linear extrapolation according to claim 9, characterized in that: The S23 is specifically: Assume that the sequence corresponding to the spatial straight line and the image square straight line is {(l1,L1),(l2,L2),...,(l N ,L N )}, where N is the number of paired lines, L and l are the spatial lines and their corresponding image-side lines, then the camera extrinsic parameter optimization problem based on the spatial lines and image-side lines is expressed as follows: Where d(l i ,L i ) represents the distance between the image-side straight line and the reprojection space straight line, and the least squares solution of the camera extrinsic parameters is obtained by minimizing the error function E; By solving the image reprojection error e l About the camera extrinsic parameter increment δ ζ Orthogonal to the straight line represents the increment δ θ The Jacobian matrix of is used to solve the straight line reprojection error; two Jacobian matrices are obtained by chain derivation rule: in, in The Jacobian matrix between the image-side reprojection error and the state variables is obtained through the above formula, and the Levenberg-Marquardt nonlinear optimization calculation method is used to achieve an accurate solution of the camera extrinsic parameters.

Images