A single scale field calibration method

Abstract

The present application discloses a single-scale on-site calibration method, which belongs to the field of camera calibration technology, including: using a straight-line scale and using a binocular camera to obtain a calibration image; relative orientation of each camera and obtaining the initial values ​​of the camera's internal and external parameters: by relative orientation, determining the relative position and posture relationship between each camera in the binocular camera, establishing a local coordinate system with the reference camera as a reference, and using the coplanar condition between the cameras to obtain the initial values ​​of the camera's internal and external parameters; absolute orientation of each camera and solving the external parameters of the calibration image obtained by each camera: by absolute orientation, determining the position and posture of the binocular camera in the world coordinate system, obtaining the world coordinates of all calibration points of the calibration image, solving the external parameter values ​​of each calibration image relative to the world coordinate system; and optimizing and adjusting the camera parameters. The present invention solves the problem of difficulty in on-site calibration of large fields of view, including some ultra-large fields of view.

Description

Technical Field

[0001] The present application relates to a single-scale on-site calibration method, belonging to the technical field of camera calibration. Background Art

[0002] Camera calibration is to accurately map objects in the three-dimensional world onto a two-dimensional image plane by determining the internal and external parameters of the camera. The traditional camera calibration method is to use a calibration plate (usually a specific black and white chessboard pattern) for calibration. By taking images of the calibration plate at different postures, the camera's internal parameters (such as focal length, principal point position, etc.) and external parameters (such as the camera's position in the world coordinate system) are calculated and optimized accordingly. There are currently many different algorithms and methods for camera calibration. For example, the most commonly used two-step method proposed by RYTsai and the chessboard grid plane calibration method proposed by Zhang Zhengyou require the production of high-precision targets to ensure the final calibration accuracy; and the algorithm module can be provided free of charge through visual algorithm libraries such as opencv or matlab, which is the most widely used calibration method in the current field of visual measurement. The camera self-calibration idea proposed by Fanugeras et al. does not require the known target Euclidean spatial structure and other spatial three-dimensional information. It only uses the constraint relationship between multiple imaging to calculate the camera parameters. At the same time, the camera's intrinsic parameters, extrinsic parameters and target three-dimensional information are all taken as unknown quantities for overall bundle adjustment. It is currently the most commonly used in industrial measurement. In addition, there is a camera calibration method based on deep learning, which can directly predict the camera's internal and external parameters by training a neural network.

[0003] The existing commonly used camera calibration methods have the following technical difficulties:

[0004] (1) The calibration plate has limitations: In order to ensure the calibration accuracy of the camera system, it is usually necessary to use a calibration plate that is the same size or similar in size to the actual measurement scene for calibration, that is, the calibration plates in different scenes cannot be universal. As the scene expands, the overall size of the calibration plate will increase accordingly; at the same time, due to the different materials used, the calibration plate will increase in weight to varying degrees, and often requires multiple people to calibrate together. In addition, the large size and thin thickness of the calibration plate make it more likely to cause surface deformation problems during actual use, so higher requirements are placed on the actual calibration plate's use environment, storage conditions, and transportation process.

[0005] (2) Complex image acquisition: The camera calibration process requires the acquisition of multiple sets of calibration plate images, which requires that the image acquisition position is accurate and that images are present at all positions of the calibration plate within the field of view. For large scenes, this undoubtedly increases the complexity and difficulty of the calibration work.

[0006] (3) High consumption of time and resources: Under the condition of large field of view, large-sized calibration plates are required. Large-sized calibration plates are heavy and often require collaborative operation by multiple people. In order to ensure the continuous acquisition of multiple calibration plate images, it is undoubtedly a huge test for the operator's physical strength. In addition, the limitations of existing materials and processing technology and huge financial investment also make it difficult to completely solve the limitations of existing calibration plates.

[0007] (4) The calibration image solution rate is low: For binocular and multi-cameras, the calibration process is not only to solve the internal and external parameters of a single camera, but also to determine and establish the relative position relationship between the two cameras. Using the calibration board as the shooting target, the three-dimensional coordinates of the marker points on the board and the coordinates of the image points need to be adjusted continuously. The large number of calibration unknowns increases the amount of calculation data and the complexity of the calculation, and the overall solution rate is very low.

[0008] Therefore, due to the limitations of the use environment of the calibration board and the complexity of image acquisition, the existing methods are not suitable for on-site calibration in large scenes. Summary of the invention

[0009] According to one aspect of the present application, a single-scale on-site calibration method is provided, which solves the problem of difficulty in on-site calibration of a large field of view, including a part of an ultra-large field of view.

[0010] A single scale on-site calibration method, characterized by comprising:

[0011] Use a straight ruler and a binocular camera to obtain a calibration image;

[0012] Relative orientation is performed on each camera and initial values ​​of internal and external parameters of the camera are obtained: by relative orientation, the relative position and posture relationship between the cameras in the binocular camera are determined, a local coordinate system with the reference camera as a reference is established, and initial values ​​of internal and external parameters of the camera are obtained by using the coplanar condition between the cameras;

[0013] Absolutely orient each camera and solve the extrinsic parameters of the calibration image obtained by each camera: determine the position and posture of the binocular camera in the world coordinate system through absolute orientation, obtain the world coordinates of all calibration points of the calibration image, and combine the three-dimensional coordinates of the local coordinate system of the marker points on each image after absolute orientation to solve the extrinsic parameter values ​​of each calibration image relative to the world coordinate system;

[0014] Optimize and adjust the camera parameters: Use the collinearity equation as the observation equation, perform full parameter bundling adjustment, and solve and output the internal and external parameter values ​​of the camera system with the constraint that the internal parameters of each camera and the relative external parameters of the non-reference camera no longer change.

[0015] Furthermore, a one-line ruler is used and a binocular camera is used to obtain a calibration image, including:

[0016] 1.1 Design marking points: dot matrix coding points are used as marking points. The marking points are composed of 8 dots with the same diameter. The points at the five positions A, B, C, D, and E are used as template points. The cross ratio invariance principle is used to locate the coding area. The other three marking points are assigned unique digital numbers according to their different positions in the design coordinate system to distinguish the overall dot matrix coding;

[0017] According to perspective geometry, the projection of the image target from the three-dimensional world coordinate system to the two-dimensional measurement coordinate system of the image follows the nonlinear perspective projection transformation; the cross ratio invariant is the invariant under perspective projection, that is:

[0018] ;

[0019] Among them, A, B, and C are collinear to obtain a straight line BC, and the intersection of the straight line BC and the line connecting E and D is F, and C r =1.5112;

[0020] 1.2 Marking point detection:

[0021] 1.2.1 Ellipse recognition: The acquired image is subjected to noise reduction, sub-pixel edge extraction and ellipse fitting in turn to obtain the coordinates of the ellipse center point and the length parameters of the major semi-axis;

[0022] 1.2.2 Clustering to generate point clusters: The obtained ellipse center points are randomly divided into multiple point clusters, and the number of points in each point cluster is not less than 8;

[0023] 1.2.3 Identify template points: Set the point closest to the center of gravity of the point cluster as template point A; set the point farthest from template point A as template point B; template point C is collinear with points AB and is located on the reverse extension line of the AB line; D and E are located on both sides of point C and the angle between them and the two vectors is not less than 20°;

[0024] 1.2.4 Calculate the cross ratio: identify five template points and bring them into the cross ratio formula, and output the corresponding cross ratio value; if the cross ratio value is 1.5112, it is dot matrix coding and proceed to the next step; otherwise, return to re-identify template point B;

[0025] 1.2.5 Calculate affine transformation parameters: According to the 2D image coordinates of the five template points and the given 2D design coordinates, solve the affine transformation parameters between the two coordinate systems;

[0026] 1.2.6 Marker point matching: Based on the obtained affine transformation parameters and the image coordinates of the other three points except the template point, the design coordinates corresponding to the three points are obtained;

[0027] 1.2.7 Solve the final code: According to the coding principle, design the coordinates to find the corresponding coding ID, and output the final ID of all marked points in the point block;

[0028] 1.2.8 Image calibration: Use a straight ruler to collect the calibration image of the binocular camera.

[0029] Furthermore, each camera is relatively oriented, including:

[0030] The coplanarity condition equation is satisfied:

[0031] ;

[0032] The first row represents the three components of the photographic baseline relative to the image space auxiliary coordinate system: , that is, the coordinates of the origin of the right image coordinate system in the auxiliary coordinate system of the image space;

[0033] The second line represents the coordinates of the left image point in the left coordinate system ;

[0034] The third line represents the coordinates of the right image point in the auxiliary coordinate system of the image space ;

[0035] According to the properties of the determinant, we can get: ;

[0036] The measurement results of the left and right image point coordinates are expressed as follows: ; ;

[0037] Will It is expressed as follows: ;

[0038] Finally, the coplanar condition equation is transformed into a nonlinear equation, namely:

[0039] ;

[0040] According to the resection method, if we want to get the relative orientation result, we first need to perform Taylor expansion on the coplanar condition equation, that is:

[0041] ;

[0042] Partial derivatives solution: ;

[0043] Divide both sides by , we get:

[0044] ;

[0045] At the same time, parallax Q is introduced: ;

[0046] in, ;

[0047] but: ;

[0048] because, ;

[0049] but: ;

[0050] Get parallax Error equation: ;

[0051] Among them, each pair of homologous image points has a set of parallax , measure more than 7 pairs of same-name image points and set the parallax to zero to complete the relative orientation of the camera; solve the relative orientation parameters and the initial values ​​of the internal parameters according to the least squares adjustment method.

[0052] Furthermore, the relative orientation parameters and the initial values ​​of the internal parameters are solved according to the least squares adjustment method, including:

[0053] Obtain known image data: determine the camera's factory focal length parameters and left image point coordinates and the right image point coordinates ;

[0054] Assumed Photographic Baseline: Assume the photographic baseline is ;

[0055] Set all parameters to zero: Assume that the initial values ​​of all relative orientation parameters are zero; the principal point distance between the two cameras is ;

[0056] Solve image space auxiliary coordinates: Calculate image space auxiliary coordinates as well as ;

[0057] Solve the error equation: calculate the coefficients and constant terms of the error equation point by point and convert them into an equation;

[0058] Iteratively determine the initial value of the element: According to the same-plane constraint relationship, the parallax equation is required to be zero, and the solution is continuously iterated. After the final iteration converges, the final relative orientation parameter value and the intrinsic parameter values ​​of the two cameras are output.

[0059] Furthermore, before absolutely orienting each camera, the following steps are also included:

[0060] The initial values ​​of relative extrinsic parameters are corrected by using the scale. The local coordinate axis vector is taken as the unit vector, the scale factor is introduced, and the corresponding relationship between the local coordinate distance and the actual physical distance is established by using the geometric relationship that the sum of the distances from the landmark point to its center of mass remains unchanged. The initial values ​​of relative extrinsic parameters of the non-reference camera are solved.

[0061] Furthermore, the initial values ​​of the relative external parameters are corrected by the scale, including:

[0062] Determine in the image space auxiliary coordinate system, In the right image Coordinates of group marker points in the auxiliary coordinate system of image space , we can see that the centroid coordinates in this image can be expressed as:

[0063] ;

[0064] Find the sum of the distances from the jth group of landmarks to the centroid in the i-th image:

[0065] ;

[0066] ;

[0067] in, represents the distance from the jth group of landmarks to the centroid in the i-th image; i represents the number of images taken, ; j represents the number of landmark points captured in the i-th image, ; Represents the sum of the distances from each group of landmark points to the centroid in the i-th image;

[0068] The sum of the distances from the markers to the centroid on each image remains unchanged:

[0069] ;

[0070] In the formula, It represents the mean value of the sum of the distances from each group of landmark points to the centroid in all images taken in the image space auxiliary coordinate system, where: Its sum is constant and remains unchanged;

[0071] According to the DP close-range industrial photogrammetry system, the global point coordinate set of all the marking points on the ruler is obtained. , assuming that the coordinates of a point in the object space are , solve the centroid coordinates of the global point and find the sum of the distances from each point to the centroid;

[0072] According to the global point coordinates, the centroid coordinates are obtained:

[0073] ;

[0074] Solve the distance from each group of landmark points to the centroid in the image :

[0075] ;

[0076] And the distance from each group of landmark points to the centroid Summing, we get :

[0077] ;

[0078] Determine the distance ratio relationship between the two groups as:

[0079] ;

[0080] Where: S represents the sum of the distances from each group of landmarks to the centroid in the world coordinate system The mean of the sum of the distances from each group of landmarks to the centroid in all images captured in the image space auxiliary coordinate system The ratio of

[0081] The proportional relationship S, in addition to being a distance ratio, also represents the proportional relationship from the image space auxiliary coordinate system to the identification coordinate system. This proportional relationship is applied to the three components of the photographic baseline relative to the image space auxiliary coordinate system: , combined with the three corner elements obtained , determine the initial values ​​of the relative external parameters of the current camera system.

[0082] Furthermore, each camera is absolutely oriented and the extrinsic parameters of the calibration image acquired by each camera are solved, including:

[0083] Determine the center of mass position;

[0084] According to the set of three-dimensional coordinate points determined in any image , where the three-dimensional coordinates of any point are , then the centroid The location is as follows: ;

[0085] Calculate the distance from each point to the centroid;

[0086] According to the three-dimensional coordinate point set and centroid The position coordinates of , determine the distance from each point to the center of mass and the sum of the distances:

[0087] ;

[0088] ;

[0089] Where: Represents a set of three-dimensional coordinate points From a point in The sum of the distances;

[0090] Represents a set of three-dimensional coordinate points All points to the centroid The sum of the distances;

[0091] Determine the proportional relationship between the images;

[0092] Combine the sum of the distances from the global point to the centroid , determine the proportional relationship between the image space auxiliary coordinate system of the left image and the world coordinate system:

[0093] ;

[0094] Where: It represents the sum of the distances from each group of landmarks to the centroid in the world coordinate system;

[0095] The proportional relationship Apply to 3D coordinate point sets , the corrected left image space is the same size as the world coordinate system, so there exists a set of rotation and translation matrix relationships that satisfy:

[0096] ;

[0097] ;

[0098] in, represents the rotation matrix relationship from the world coordinate system to the image space auxiliary coordinate system, and T represents the translation matrix relationship from the world coordinate system to the image space auxiliary coordinate system;

[0099] The two coordinate sets satisfy the SVD singular value decomposition method, so the matrix transformation relationship between the two coordinate sets is solved:

[0100] Solve for the matrix centroid;

[0101] Let matrix Their respective centroids are ,but:

[0102] ;

[0103] Translation matrix;

[0104] The matrix , They are translated relative to their respective center of mass positions, and the new matrices are , ,but: ;

[0105] SVD decomposition; using matrix , Constructing the Matrix , and perform SVD decomposition on it: ;

[0106] Solve ; ; Using SVD singular value decomposition method, we can get , determine the external parameters of each image relative to the world coordinate system.

[0107] Furthermore, the camera parameters are optimized and adjusted, including:

[0108] Use the error equation as the observation equation for all images of the left and right cameras respectively;

[0109] For the baseline camera: ;

[0110] in: Residual represents the coordinates of the image point; are the correction numbers of the intrinsic parameters, extrinsic parameters and 3D coordinates of the reference camera respectively; They are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters and object point coordinates of the reference camera respectively; To observe the deviation between the true value and the initial value;

[0111] in: ;

[0112] Also for non-reference cameras, there is an error equation: ;

[0113] in: represents the partial derivative matrix of the intrinsic parameters of the non-reference camera; represents the intrinsic parameters of the non-reference camera; Represents the three-dimensional coordinates of object points; Represents the partial derivative matrix of the image point coordinates of the non-reference camera relative to the reference camera extrinsic parameters,

[0114] ; represents the partial derivative matrix of the image point of the non-reference camera with respect to its extrinsic parameters, ; Represents the correction number relative to the external parameters, ; Represents the partial derivative matrix corresponding to the object point coordinates of the non-reference camera,

[0115] ; Determine the global point coordinates of the marked point, that is, the world coordinates of the marked point;

[0116] When the intrinsic parameters of the camera and the relative extrinsic parameters of the non-reference camera do not change, the final intrinsic and extrinsic parameters of the camera and the relative extrinsic parameter values ​​of the non-reference camera are output, and the intrinsic and extrinsic parameter calibration of the camera system is completed.

[0117] Furthermore, the straight ruler is a carbon fiber straight ruler;

[0118] The marking points adopt a structure of a metal base film and reflective micro-bead dots, and the reflectivity of each marking point is 300 mesh to 400 mesh;

[0119] When the straight ruler is used in the horizontal and vertical directions, the straight ruler moves up, down, left and right in a fixed plane; when calibrating the depth of field direction, the front depth of field and the back depth of field positions are photographed simultaneously, and the marking point of the straight ruler in the depth of field direction is inclined at an angle of 45° to the ruler body of the straight ruler.

[0120] The beneficial effects of this application include:

[0121] 1) The single-scale on-site calibration method provided in this application preliminarily determines the position relationship of the cameras through relative orientation, and continues to use the same-planarity constraint condition to obtain the camera's internal and external parameters and the initial value of the right camera's relative external parameters through least squares optimization iteration; solves the external parameters of each image through absolute orientation: absolute orientation is achieved through point cloud registration combined with SVD (singular value decomposition method), and the camera parameters are unified in the coordinate system. The relationship between the left image space and the world coordinate system is the external parameter of the left camera, and it is also the external parameter of the camera system under the relative external parameter model. Since the external parameter relationship will change with the position of the image, scale correction is performed before absolute orientation, that is, the proportional relationship between the left image space and the world coordinate system is established; then the matrix transformation relationship between each image and the world coordinate system is determined by using point cloud registration combined with SVD (singular value decomposition method); all parameters under the camera relative external parameter model are adjusted, with the camera error equation as the observation equation, and the convergence of the camera internal parameters and relative external parameters as constraints. The actual final calibration result is accurate and efficient. During the adjustment process, no calibration parameters are added in large quantities and each parameter is assigned a corresponding initial value or actual parameter value. Therefore, the equation solution can converge quickly to obtain the calibration parameter results, solving the problem of difficult on-site calibration of large fields of view, including some ultra-large fields of view.

[0122] 2) The single-scale on-site calibration method provided in this application uses a carbon fiber straight-line scale. The adjusted scale reduces the difficulty of operation technology and shortens the calibration time. The scale body uses a carbon fiber structure, which greatly reduces the weight of the scale; and the metal film is combined with the reflective dots of glass beads to design the marking points, which reduces the difficulty of technical operation and ensures the recognition effect of the marking points. BRIEF DESCRIPTION OF THE DRAWINGS

[0123] Figure 1 This is a flow chart of a single-scale on-site calibration method in one embodiment of the present application;

[0124] Figure 2 This is a camera imaging principle diagram in one embodiment of the present application;

[0125] Figure 3 A dot matrix coding diagram in one embodiment of the present application;

[0126] Figure 4 This is an effect diagram of the calibration image acquisition process in one implementation of the present application;

[0127] Figure 5 This is a relative orientation principle diagram in one embodiment of the present application (the left camera is fixed);

[0128] Figure 6 This is a photographic baseline illustration in one embodiment of the present application;

[0129] Figure 7 This is a parallax schematic diagram in one implementation of the present application;

[0130] Figure 8 This is a dot matrix coding design diagram in one embodiment of the present application;

[0131] Fig. 9 This is a design drawing of a one-line ruler in one embodiment of the present application;

[0132] Fig.10 This is a tiling diagram of a one-line ruler in one embodiment of the present application;

[0133] Fig.11 In one embodiment of the present application, the depth of field feature point Figure 1 ;

[0134] Fig.12 In one embodiment of the present application, the depth of field feature point Figure 2 . DETAILED DESCRIPTION

[0135] The present application is described in detail below with reference to embodiments, but the present application is not limited to these embodiments.

[0136] See also Figure 1-12 ,like Figure 1 As shown, a single scale on-site calibration method is characterized by comprising:

[0137] Use a straight ruler and a binocular camera to obtain a calibration image;

[0138] Relative orientation is performed on each camera and initial values ​​of internal and external parameters of the camera are obtained: by relative orientation, the relative position and posture relationship between the cameras in the binocular camera are determined, a local coordinate system with the reference camera as a reference is established, and initial values ​​of internal and external parameters of the camera are obtained by using the coplanar condition between the cameras;

[0139] Absolutely orient each camera and solve the extrinsic parameters of the calibration image obtained by each camera: determine the position and posture of the binocular camera in the world coordinate system through absolute orientation, obtain the world coordinates of all calibration points of the calibration image, and combine the three-dimensional coordinates of the local coordinate system of the marker points on each image after absolute orientation to solve the extrinsic parameter values ​​of each calibration image relative to the world coordinate system;

[0140] Optimize and adjust the camera parameters: Use the collinearity equation as the observation equation, perform full parameter bundling adjustment, and solve and output the internal and external parameter values ​​of the camera system with the constraint that the internal parameters of each camera and the relative external parameters of the non-reference camera no longer change.

[0141] Specifically, the image acquisition in the existing image calibration is complex. The camera calibration process needs to collect multiple sets of calibration plate images, requiring the image acquisition position to be accurate and the calibration plate to have images at all positions within the field of view. For large scenes, this undoubtedly increases the complexity and difficulty of the calibration work. At the same time, time and resource consumption is large. Under the condition of a large field of view, a large-sized calibration plate is required. The large-sized calibration plate is heavy and often requires collaborative operation by multiple people. In order to ensure the continuous acquisition of multiple calibration plate images, it is undoubtedly a huge test for the operator's physical strength. In addition, the existing materials and processing technology are limited, and huge financial investment is also difficult to completely solve the limitations of the existing calibration plate. The calibration image solution rate is low. For binocular and multi-eye cameras, the calibration process is not only the solution of the internal and external parameters of a single camera, but also the determination and establishment of the relative position relationship between the two cameras. Taking the calibration plate as the shooting target, the three-dimensional coordinates of the marker points on the plate and the image point coordinates need to be continuously adjusted. A large number of calibration unknowns increase the amount of calculation data and the complexity of calculation, and the overall solution rate is very low.

[0142] Therefore, the present application adopts a straight ruler. Compared with a flat surface, the straight narrow strip shape not only reduces the material cost, but also reduces the overall weight, and is also convenient for transportation and operation. Using dot matrix reflective markers, marker point detection is a very critical step in image processing, and the quality of the detection effect directly determines the accuracy of the subsequent calibration results. If you want to improve the marker point detection effect, improving the detection algorithm is a good technical means, but improving image quality is a more direct way. Furthermore, the initial values ​​of the internal and external parameters of the camera are obtained through relative orientation. Through relative orientation, the relative position and posture relationship between each camera in the camera system can be determined, and a local coordinate system with the reference camera as the reference is established. Using the coplanar condition between the cameras, an error equation containing the relationship between the internal and external parameters of the camera is introduced for each pair of homologous points, and the initial values ​​of the internal and external parameters of the camera are obtained by solving. The introduction of the local coordinate system reduces the amount of image data and simplifies the data solution process. By solving the external parameters of each image through absolute orientation, the absolute orientation can determine the position and posture of the camera system in the world coordinate system, that is, based on the DP close-range industrial photogrammetry system to obtain the world coordinates of all landmarks, combined with the local coordinate system three-dimensional coordinates of the landmarks on each image after relative orientation, the external parameter values ​​of each image relative to the world coordinate system are solved based on point cloud registration and SVD (singular value decomposition) method. The above process determines the initial values ​​of the intrinsic parameters of each camera, the initial values ​​of the relative external parameters of the non-reference camera, and the external parameters of each image relative to the world coordinate system in turn, and uses the collinear equation as the observation equation to perform full parameter bundling adjustment, and solves and outputs the internal and external parameter values ​​of the camera system with the constraint that the intrinsic parameters of each camera and the relative external parameters of the non-reference camera no longer change.

[0143] It is worth noting that in close-range photogrammetry, camera imaging can be approximately represented by a pinhole imaging model. The actual camera imaging model is as follows: Figure 2 As shown, in Figure 2 In the object point , passing through the lens optical center After that, the image point on the image plane .in, is the principal point of the image plane, The distance is the focal length . — Coordinates of the object point in the world coordinate system / mm; —Coordinate of the lens optical center in the world coordinate system / mm; —The image point coordinates / pixel corresponding to the object point; —Principal point deviation / pixel; —focal length / mm; —Image point deviation / pixel caused by lens distortion.

[0144] ;

[0145] focal length , principal point deviation Image point deviation caused by lens distortion is the intrinsic parameter of the camera; and They respectively represent the rotation and translation matrix relationship from the world coordinate system to the camera coordinate system, that is, the external parameters of the camera.

[0146] The actual image collinearity equation is as follows:

[0147] ;

[0148] in: —Coordinates of the object point in the world coordinate system / mm; —Coordinate of the lens optical center in the world coordinate system / mm; —The image point coordinates / pixel corresponding to the object point; —Principal point deviation / pixel; —Focal length / mm; —Image point deviation / pixel caused by lens distortion.

[0149] ;

[0150] ;

[0151] focal length , principal point deviation and image point deviation caused by lens distortion; is the intrinsic parameter of the camera; and They respectively represent the rotation and translation matrix relationship from the world coordinate system to the camera coordinate system, that is, the external parameters of the camera.

[0152] The traditional visual system calibration outputs the internal and external parameters of each camera, without considering the positional relationship between cameras. Therefore, it is proposed that in a multi-camera visual system, the positional relationship of cameras can be constrained by relative external parameters. This application takes a binocular camera system as an example:

[0153] Define one of the cameras as the reference camera, and the other camera as the non-reference camera. The non-reference camera has a relative position relationship with the reference camera, which can be expressed as a set of relative external parameters:

[0154] ;

[0155] For the reference camera, the collinearity equation is Taylor expanded and subdivided into the error equation:

[0156] ;

[0157] in: Residual represents the coordinates of the image point; are the correction numbers of the intrinsic parameters, extrinsic parameters and 3D coordinates of the reference camera respectively; They are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters of the reference camera and the three-dimensional coordinates of the object point; is the deviation between the observed true value and the initial value.

[0158] Also for non-reference cameras, there is an error equation:

[0159] ;

[0160] in: Represents the partial derivative matrix of the image point coordinates of the non-reference camera relative to the reference camera extrinsic parameters, Represents the partial derivative matrix of the image point of the non-reference camera with respect to its extrinsic parameters; Indicates the correction number relative to the external parameters.

[0161] Since the collinearity equation of the non-reference camera removes the six independent extrinsic parameters of each image and introduces six relative extrinsic parameters, the unknowns of the bundle adjustment can be reduced in the process of solving the parameters of the non-reference camera. , among which is the number of images.

[0162] Using a ruler and a binocular camera to obtain calibration images includes:

[0163] 1.1 Design marking points: dot matrix coding points are used as marking points. The marking points are composed of 8 dots with the same diameter. The points at the five positions A, B, C, D, and E are used as template points. The cross ratio invariance principle is used to locate the coding area. The other three marking points are assigned unique digital numbers according to their different positions in the design coordinate system to distinguish the overall dot matrix coding;

[0164] According to perspective geometry, the projection of the image target from the three-dimensional world coordinate system to the two-dimensional measurement coordinate system of the image follows the nonlinear perspective projection transformation; the cross ratio invariant is an invariant under perspective projection, and the cross ratio is defined according to the distance between four points on the same straight line, that is, the straight line The cross ratio definition is:

[0165] ;

[0166] Among them, A, B, and C are collinear to obtain a straight line BC, and the intersection of the straight line BC and the line connecting E and D is F, and C r=1.5112, which also satisfies the cross-ratio invariance principle. The three marking points used to distinguish the codes are distributed in 20 design positions, and any two points cannot be adjacent. It is worth noting that A', B', C', D', E', and F' are the projection points of A, B, C, D, E, and F respectively;

[0167] Specifically, as a commonly used detection object in visual detection, in order to accurately and quickly extract landmark points, this application uses dot matrix coding points as marking points.

[0168] Its advantages are: 1) High resolution rate: using the cross ratio constant feature, the dot matrix coding area can be quickly locked; 2) Large amount of coding: by changing the position of some dots, different codes can be distinguished. Figure 3 As shown, Figure 3 (a) is the dot matrix coding diagram, Figure 3 (b) is the cross ratio definition diagram, a set of dot matrix codes consists of 8 dots with the same diameter. The points at the five positions are used as template points, and the coding area is located by using the cross ratio invariance principle; in addition, the other three marking points are assigned unique digital numbers according to their different positions in the design coordinate system, that is, the three marking points are used to distinguish the overall dot matrix coding, such as Figure 3 As shown in the figure, a set of dot matrix codes consists of 8 dots with the same diameter. The three marking points (three points other than the five points A, B, C, D, E, and F in the figure) used to distinguish the dot matrix codes are required to be distributed in 20 design positions, and any two points cannot be adjacent. The points at five positions are used as template points. Three points collinear, D,E Make another connection straight line ABC and The intersection point of the connecting lines is determined as , defined by perspective geometry, the intersection ratio for: .

[0169] 1.2 Marking point detection:

[0170] 1.2.1 Ellipse recognition: The acquired image is subjected to noise reduction, sub-pixel edge extraction and ellipse fitting in turn to obtain the coordinates of the ellipse center point and the length parameters of the major semi-axis;

[0171] 1.2.2 Clustering to generate point clusters: The obtained ellipse center points are randomly divided into multiple point clusters. The point differences between different clusters are as large as possible, and the number of points in each point cluster is not less than 8;

[0172] 1.2.3 Identify template points: Set the point closest to the center of gravity of the point cluster as template point A; set the point farthest from template point A as template point B; template point C is collinear with points AB and is located on the reverse extension line of the AB line; D and E are located on both sides of point C and the angle between them and the two vectors is not less than 20°;

[0173] Specifically, identify the template points:

[0174] 1) Determine the template point A by the centroid: From the geometric relationship, we know that the template point A should be located at the centroid of the point cluster, so the point closest to the centroid of the point cluster is set as the template point A;

[0175] 2) The point farthest from point A is template point B: According to the definition of intersection ratio, template point B should be the farthest from point A in the point cluster relationship, so the point farthest from template point A is set as template point B;

[0176] 3) AB line determines template point C: From the definition of intersection ratio, point C is collinear with points AB and is located on the reverse extension line of AB line, so in vector Find point C on the reverse extension line of

[0177] 4) Points D and E are located on both sides of point C: Based on the above principle, points D and E should be located on both sides of point C and and The angle between the two vectors is not less than 20°, so find the appropriate points D and E in turn.

[0178] At this point, the template points are identified and determined in turn. It should be noted that in the above four point search steps, if one of the steps cannot meet the preset conditions, it is necessary to return to the previous step to find other point clusters that meet the conditions.

[0179] 1.2.4 Calculate the cross ratio: identify five template points and bring them into the cross ratio formula, and output the corresponding cross ratio value; if the cross ratio value is 1.5112, it is dot matrix coding and proceed to the next step; otherwise, return to re-identify template point B;

[0180] 1.2.5 Calculate affine transformation parameters: According to the 2D image coordinates of the five template points and the given 2D design coordinates, solve the affine transformation parameters between the two coordinate systems;

[0181] 1.2.6 Marker point matching: Based on the obtained affine transformation parameters and the image coordinates of the other three points except the template point, the design coordinates corresponding to the three points are obtained;

[0182] 1.2.7 Solve the final code: According to the coding principle, design the coordinates to find the corresponding coding ID, and output the final ID of all marked points in the point block;

[0183] 1.2.8 Image calibration: Use a straight ruler to collect the calibration image of the binocular camera.

[0184] Specifically, dot matrix coded marking points are designed and used as scale feature targets. This application takes the binocular system as an example. The shooting principles are: (1) the black and white contrast in the image is obvious, and the feature points are clearly visible; (2) the left and right camera images are ensured to occupy the entire camera field of view and depth of field; (3) during the entire shooting process, the camera system does not have obvious displacement or vibration.

[0185] Among them, the calibration image acquisition process is as follows Figure 4 As shown, Figure 4 (a) refers to vertical calibration, Figure 4 (b) refers to horizontal calibration, Figure 4 (c) Refers to the depth of field direction calibration.

[0186] This application uses a ruler to collect calibration images of the binocular system. First, after fixing the camera system, adjust the lens focal length according to the measurement distance to ensure image quality; then, within the field of view of the fixed measurement distance, move the ruler vertically and horizontally and collect the ruler image in each position; then, within the field of view corresponding to the foreground depth of field and the back depth of field distance of the camera, move the ruler vertically and horizontally and collect the ruler image in each position; finally, transmit the ruler image to the computer for subsequent image processing.

[0187] Relative orientation usually refers to determining the relative position and direction relationship between multiple cameras, as well as the positioning and orientation information relative to a reference coordinate system;

[0188] Assuming that the left image space coordinate system is used as the image space auxiliary coordinate system and coincides with the world coordinate system, the six extrinsic parameters of the left image are zero; and the extrinsic parameters of the right image become the relative position and attitude parameters of the right image space coordinate system relative to the left image space coordinate system, that is, the three components of the photographic baseline and the three angular elements of the right image. Among them, the relative position and running attitude parameters between the two image space coordinate systems of the stereo image pair are: three components - the three components of the photographic baseline relative to the image space auxiliary coordinate system ; Three external parameters - the three angular elements of the right image space coordinate system relative to the image space auxiliary coordinate system, such as Figure 5 The three corners marked on the right ,like Figure 5As shown, relative orientation is used to determine the relative position and direction relationship between multiple cameras, as well as the positioning and orientation information relative to a reference coordinate system. In the figure, S1 represents the origin of the left image space coordinate system, S2 represents the origin of the right image space coordinate system, and the photographic baseline B. Taking the left image space coordinate system S1-x1y1z1 as the auxiliary image space coordinate system, and coinciding with the world coordinate system, the six extrinsic parameters of the left image are zero. The extrinsic parameters of the right image become the relative position and posture parameters of the right image space coordinate system S2-x2y2z2 relative to the left image space coordinate system S1-x1y1z1, including: the coordinates of the origin S2 of the right image coordinate system in the auxiliary image space coordinate system , which are also the three components of the photographic baseline B relative to the auxiliary coordinate system of the image space And the three angular elements of the right image space coordinate system relative to the image space auxiliary coordinate system , (X1, Y1, Z1) represents the three-dimensional spatial coordinates of the image point of the left image in the left image space coordinate system S1-x1y1z1; (X2, Y2, Z2) represents the three-dimensional spatial coordinates of the image point of the right image with the same name as the left image in the right image space coordinate system S2-x2y2z2; A(X, Y, Z) represents the three-dimensional spatial coordinates of the real object point corresponding to the left and right image points in the world coordinate system;

[0189] The relative orientation satisfies the coplanarity condition equation, that is, theoretically, rays of the same name intersect in the same epipolar plane, and the vector mixed product of the three lines is equal to zero. According to the coplanarity condition equation, if three points can be found on three lines, the coplanarity condition equation can be used.

[0190] Relative orientation of each camera, including:

[0191] The coplanarity condition equation is satisfied:

[0192] ;

[0193] The first row represents the three components of the photographic baseline relative to the image space auxiliary coordinate system: , that is, the coordinates of the origin of the right image coordinate system in the auxiliary coordinate system of the image space;

[0194] The second line represents the coordinates of the left image point in the left coordinate system (points on the left line);

[0195] The third line represents the coordinates of the right image point in the auxiliary coordinate system of the image space (point on the right line);

[0196] According to the properties of the determinant, we can get:

[0197] ;

[0198] The measurement results of the left and right image point coordinates are expressed as follows:

[0199] ;

[0200] Among them, the three components of the photographic baseline can be expressed as follows Figure 6 As shown;

[0201] exist Figure 6 The three components of the photographic baseline relative to the image space auxiliary coordinate system are Through the solid angle To express the relationship between each other. Among them: ;

[0202] Will It is expressed as follows:

[0203] ;

[0204] Finally, the coplanar condition equation is transformed into a nonlinear equation, namely:

[0205] ;

[0206] According to the resection method, if we want to get the relative orientation result, we first need to perform Taylor expansion on the coplanar condition equation, that is:

[0207] ;

[0208] Partial derivatives solution:

[0209] ;

[0210] Divide both sides of the above equation by , we get:

[0211] ;

[0212] At the same time, after a stereo pair completes relative orientation, the upper and lower parallax of its orientation point model is zero, and the parallax Q is introduced:

[0213] ; For the same source points, it is very likely that there are two image points that do not start from the same point but are near the point, that is, the light rays of the two image points cannot intersect at the same point, which is parallax. The definition of parallax is as follows Figure 7 As shown;

[0214] exist Figure 7 In the figure, the parallax Q is caused by the fact that the light rays from the two image points a1 and a2 on the left and right cannot intersect at the same point A. Define Q: ; represents the disparity of the left image; represents the disparity of the right image; Represents the Y-axis component of the photographic baseline B relative to the auxiliary coordinate system of the image space;

[0215] in, ;

[0216] but: ;

[0217] because, ;

[0218] but: ;

[0219] Get parallax Error equation:

[0220] ;

[0221] Among them, each pair of homologous image points has a set of parallax , measure more than 7 pairs of image points with the same name and set the parallax to zero to complete the relative orientation of the camera;

[0222] The relative orientation parameters and initial values ​​of internal parameters are solved using the least squares adjustment method.

[0223] Solving the relative orientation parameters and the initial values ​​of the internal parameters using the least squares adjustment method includes:

[0224] Obtain known image data: determine the camera's factory focal length parameters and left image point coordinates and the right image point coordinates ;

[0225] Assumed Photographic Baseline: Assume the photographic baseline is ;

[0226] Set all parameters to zero: Assume that the initial values ​​of all relative orientation parameters are zero; the principal point distance between the two cameras is ;

[0227] Solve image space auxiliary coordinates: Calculate image space auxiliary coordinates as well as ;

[0228] Solve the error equation: calculate the coefficients and constant terms of the error equation point by point and convert them into an equation;

[0229] Iteratively determine the initial value of the element: According to the same-plane constraint relationship, the parallax equation is required to be zero, and the solution is continuously iterated. After the final iteration converges, the final relative orientation parameter value and the intrinsic parameter values ​​of the two cameras are output.

[0230] Before absolutely orienting each camera, the following is also included:

[0231] The initial values ​​of relative extrinsic parameters are corrected by using the scale. The local coordinate axis vector is taken as the unit vector, the scale factor is introduced, and the corresponding relationship between the local coordinate distance and the actual physical distance is established by using the geometric relationship that the sum of the distances from the landmark point to its center of mass remains unchanged. The initial values ​​of relative extrinsic parameters of the non-reference camera are solved.

[0232] Specifically, according to the above algorithm flow, the relative orientation of the image points is completed, the initial values ​​of the camera's intrinsic parameters and the initial values ​​of the relative extrinsic parameters of the non-reference camera after relative orientation are determined. The image space auxiliary coordinate system needs to be translated to the ground according to the photographic scale to establish the photogrammetric coordinate system of the model, which is used as the world coordinate system to express the coordinates of the object point model, that is, the photographic baseline needs to be corrected.

[0233] Correcting the initial values ​​of relative external parameters by scale includes:

[0234] Determine in the image space auxiliary coordinate system, In the right image The coordinates of the first group of marker points in the auxiliary coordinate system of the image space. The coordinates of the first group of marker points in the auxiliary coordinate system of the image space in the right image. , we can see that the centroid coordinates in this image can be expressed as:

[0235] ;

[0236] Find the sum of the distances from the jth group of landmarks to the centroid in the i-th image:

[0237] ;

[0238] ;

[0239] in, represents the distance from the jth group of landmarks to the centroid in the i-th image; i represents the number of images taken, ; j represents the number of landmark points captured in the i-th image, ; Represents the sum of the distances from each group of landmark points to the centroid in the i-th image;

[0240] The sum of the distances from the markers to the centroid on each image remains unchanged:

[0241] ;

[0242] In the formula, It represents the mean value of the sum of the distances from each group of landmark points to the centroid in all images taken in the image space auxiliary coordinate system, where: Its sum is constant and remains unchanged;

[0243] According to the DP close-range industrial photogrammetry system, the global point coordinate set of all the marking points on the ruler is obtained. , assuming that the coordinates of a point in the object space are , solve the centroid coordinates of the global point and find the sum of the distances from each point to the centroid;

[0244] According to the global point coordinates, the centroid coordinates are obtained:

[0245] ;

[0246] Solve the distance from each group of landmark points to the centroid in the image :

[0247] ;

[0248] And the distance from each group of landmark points to the centroid Summing, we get :

[0249] ;

[0250] Determine the distance ratio relationship between the two groups as:

[0251] ;

[0252] Where: S represents the sum of the distances from each group of landmarks to the centroid in the world coordinate system The mean of the sum of the distances from each group of landmarks to the centroid in all images captured in the image space auxiliary coordinate system The ratio of

[0253] The proportional relationship S, in addition to being a distance ratio, also represents the proportional relationship from the image space auxiliary coordinate system to the identification coordinate system. This proportional relationship is applied to the three components of the photographic baseline relative to the image space auxiliary coordinate system: , combined with the three corner elements obtained , determine the initial value of the relative external parameters of the current camera system, wherein the relative orientation parameter value obtained in the present application includes three angle elements, whose initial value is zero.

[0254] Specifically, the scale corrects the initial values ​​of the relative extrinsic parameters. The local coordinate axis vector is taken as the unit vector. In order to ensure the consistency and geometric accuracy of the local coordinate system and the world coordinate scale, a scale factor is introduced. The geometric relationship that the sum of the distances from the landmark point to its center of mass remains unchanged is used to establish the correspondence between the local coordinate distance and the actual physical distance, and the initial values ​​of the relative extrinsic parameters of the non-reference camera are solved.

[0255] Among them, the DP (Digital Photography) close-range industrial photogrammetry system adopts digital close-range industrial photogrammetry technology, which is based on the principles of photogrammetry and computer vision. It uses high-precision cameras and multi-view shooting, combined with image processing, feature extraction, 3D reconstruction and other algorithms to achieve high-precision 3D measurement of the measured object. The system has the characteristics of portability, high precision, and non-contact, and is particularly suitable for the measurement of complex structures or curved surfaces.

[0256] After relative orientation, determine the three-dimensional coordinate point set of the object points corresponding to each left image in the auxiliary coordinate system of the image space , a set of three-dimensional coordinate points It is impossible to ensure that the size is completely consistent with the world coordinate system. Therefore, before solving the external parameters of each image, the proportional relationship between the two coordinate systems is first determined.

[0257] Absolutely orient each camera and solve the extrinsic parameters of the calibration image acquired by each camera, including:

[0258] Determine the center of mass position;

[0259] According to the set of three-dimensional coordinate points determined in any image Among them, the three-dimensional coordinates of any point , then the centroid The location is as follows:

[0260] ;

[0261] Calculate the distance from each point to the centroid;

[0262] According to the three-dimensional coordinate point set and centroid The position coordinates of , determine the distance from each point to the center of mass and the sum of the distances:

[0263] ;

[0264] ;

[0265] Where: Represents a set of three-dimensional coordinate points From a point in The sum of the distances;

[0266] Represents a set of three-dimensional coordinate points All points to the centroid The sum of the distances;

[0267] Determine the proportional relationship between the images;

[0268] Combine the sum of the distances from the global point to the centroid , determine the proportional relationship between the image space auxiliary coordinate system of the left image and the world coordinate system:

[0269] ;

[0270] Where: It represents the sum of the distances from each group of landmarks to the centroid in the world coordinate system;

[0271] The proportional relationship Apply to 3D coordinate point sets , the corrected left image space is the same size as the world coordinate system, so there exists a set of rotation and translation matrix relationships that satisfy:

[0272] ;

[0273] ;

[0274] in, Represents the rotation matrix relationship from the world coordinate system to the image space auxiliary coordinate system, Represents the translation matrix relationship from the world coordinate system to the image space auxiliary coordinate system;

[0275] SVD (Singular Value Decomposition) is a mathematical tool widely used in many fields such as signal processing, statistics, linear algebra, machine learning, etc. It can decompose a matrix into three specific matrix products, thereby revealing important properties of the matrix, such as the rank of the matrix, singular values, etc.

[0276] The two coordinate sets satisfy the SVD singular value decomposition method, so the matrix transformation relationship between the two coordinate sets is solved:

[0277] Solve for the matrix centroid;

[0278] Assume matrix, Their respective centroids are ,but:

[0279] ;

[0280] Translation matrix;

[0281] The matrix They are translated relative to their respective center of mass positions, and the new matrices are ,but:

[0282] ;

[0283] SVD decomposition;

[0284] Using the Matrix Constructing the Matrix , and perform SVD decomposition on it:

[0285] ;

[0286] Solve ;

[0287] ;

[0288] Using SVD singular value decomposition method, we can get , determine the external parameters of each image relative to the world coordinate system.

[0289] Optimize and adjust the camera parameters, including:

[0290] Use the error equation as the observation equation for all images of the left and right cameras respectively;

[0291] For the baseline camera:

[0292] ;

[0293] in: Residual represents the coordinates of the image point; are the correction numbers of the intrinsic parameters, extrinsic parameters and 3D coordinates of the reference camera respectively; They are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters and object point coordinates of the reference camera respectively; To observe the deviation between the true value and the initial value;

[0294] in:

[0295] ;

[0296] Also for non-reference cameras, there is an error equation:

[0297] ;

[0298] in: represents the partial derivative matrix of the intrinsic parameters of the non-reference camera;

[0299] represents the intrinsic parameters of the non-reference camera; Represents the three-dimensional coordinates of object points;

[0300] Represents the partial derivative matrix of the image point coordinates of the non-reference camera relative to the reference camera extrinsic parameters, ;

[0301] represents the partial derivative matrix of the image point of the non-reference camera with respect to its extrinsic parameters, ;

[0302] Represents the correction number relative to the external parameters, ;

[0303] Represents the partial derivative matrix corresponding to the object point coordinates of the non-reference camera, ;

[0304] Determine the global point coordinates of the marked point, that is, the world coordinates of the marked point;

[0305] When the intrinsic parameters of the camera and the relative extrinsic parameters of the non-reference camera do not change, the final intrinsic and extrinsic parameters of the camera and the relative extrinsic parameter values ​​of the non-reference camera are output, and the intrinsic and extrinsic parameter calibration of the camera system is completed.

[0306] The global point coordinates of the marker point, that is, the world coordinates of the marker point, can be determined through the DP close-range industrial photogrammetry system. Combining the initial values ​​of the internal and external parameters of the left and right cameras and the initial values ​​of the relative external parameters, when the internal parameters of the camera and the relative external parameters of the non-reference camera do not change, the final internal and external parameters of the camera and the relative external parameters of the non-reference camera are output. At this point, the internal and external parameter calibration of the camera system is completed.

[0307] The straight ruler is a carbon fiber straight ruler;

[0308] The marking points adopt a structure of a metal base film and reflective micro-bead dots, and the reflectivity of each marking point is 300 mesh to 400 mesh;

[0309] When the straight ruler is used in the horizontal and vertical directions, the straight ruler moves up, down, left and right in a fixed plane; when calibrating the depth of field direction, the front depth of field and the back depth of field positions are photographed simultaneously, and the marking point of the straight ruler in the depth of field direction is inclined at an angle of 45° with the ruler body of the straight ruler.

[0310] Specifically, the calibration plate process requirements in the existing image calibration are high. The detection and matching accuracy of the feature points on the calibration plate directly determines the calibration accuracy of the camera system, so higher process requirements are put forward for the roundness, grayscale contrast and arrangement of the feature points. The calibration plate has limitations. In order to ensure the calibration accuracy of the camera system, it is usually necessary to use a calibration plate that is as large or close to the actual measurement scene for calibration, that is, the calibration plate in each scene cannot be universal. As the scene expands, the overall size of the calibration plate will increase accordingly; at the same time, due to the different materials used, the calibration plate will increase in weight to varying degrees, and it often requires multiple people to calibrate together. In addition, the large size and thin thickness of the calibration plate make it more likely to cause surface deformation problems during actual use, so the actual calibration plate has higher requirements for the use environment, storage conditions and transportation process. This application designs and adopts dot matrix reflective marking points. The dot matrix structure can regularly arrange as many marking points as possible in a fixed space; in addition, the use of reflective materials to make marking points reduces the requirements for the on-site environment. When the on-site environment is very complex, the exposure is reduced, and the effective recognition of the marking points can also be ensured.

[0311] Using the cross ratio invariance, this application uses dot matrix coding as the feature marker point. The dot matrix coding consists of two parts: template points and coding points. The five template points are defined according to the cross ratio to lock the landmark point clusters that meet the requirements; the remaining three coding points are determined by applying the affine transformation relationship of the template point solution, and the numbers of the three points are determined according to the numbers corresponding to the design coordinates. This process has a high success rate and a fast detection rate. Figure 8 As shown in (a), the arrangement positions of the coding points in the design coordinate system are as follows Figure 8 (b) As shown in Figure 8 In the figure, there are 8 points in total on the dot matrix coding diagram on the single scale. A, B, C, D, E, and F are template points used to determine the cross ratio value; the remaining three points other than the five template points A, B, C, D, and E are used to distinguish the dot matrix number. The three points used to distinguish the dot matrix number may exist in 1 to 20 positions.

[0312] Among them, the design coordinates of each point in the dot matrix coding design diagram are shown in Table 1:

[0313] Table 1

[0314]

[0315] Since the marking point detection process requires the extraction and fitting of the ellipse edge, good image quality is very important for feature extraction. In order to better realize the feature extraction work, the present invention uses reflective materials made of glass beads to make marking points, and the reflectivity of a single reflective marking point can be guaranteed to be between 300 mesh and 400 mesh. Under the same experimental conditions, the brightness of the reflective marking points is significantly better than that of the marking points made of ordinary materials. In addition, for complex large-field-of-view environments, in order to ensure image brightness, there are often noise points that affect feature recognition; and the process of reflective points can reduce the overall image exposure time while ensuring the recognition of marking points.

[0316] At the same time, in a wide field of view environment, the calibration object is large in size and heavy in weight, and it often requires the collaboration of multiple people to complete the calibration. In response to this problem, the present application specifically proposes the use of a carbon fiber ruler structure. As a high-strength, low-density material, carbon fiber has high strength, good rigidity, light weight and corrosion resistance, and is widely used in aerospace, automotive energy, construction and other fields. The one-line ruler designed by the present invention uses a carbon fiber structure for the ruler body, and a metal film with reflective micro-bead dot structure for the surface marking points. Fig. 9 is the depth of field direction scale diagram, Fig.10 It is a horizontal and vertical ruler diagram. As can be seen from the calibration diagram in the previous section, when using the ruler in the horizontal and vertical directions, the ruler moves up and down, left and right in a fixed plane; and when calibrating the depth of field direction, since the foreground depth of field and the back depth of field positions are shot at the same time, the marking points of the ruler in the depth of field direction are not parallel to the plane structure of the ruler body, but are tilted at a 45° angle to the ruler body. Figure 11-12 The design of the entire ruler, including the ruler body and surface marking points, has been designed and optimized for a large field of view environment. While ensuring overall rigidity and strength, it reduces the weight of the ruler and reduces the calibration operation requirements.

[0317] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.

Claims

1. A single scale on-site calibration method, characterized in that: include: Use a straight ruler and a binocular camera to obtain a calibration image; Relative orientation is performed on each camera and initial values ​​of internal and external parameters of the camera are obtained: by relative orientation, the relative position and posture relationship between the cameras in the binocular camera are determined, a local coordinate system with the reference camera as a reference is established, and initial values ​​of internal and external parameters of the camera are obtained by using the coplanar condition between the cameras; Absolutely orient each camera and solve the extrinsic parameters of the calibration image obtained by each camera: determine the position and posture of the binocular camera in the world coordinate system through absolute orientation, obtain the world coordinates of all calibration points of the calibration image, and combine the three-dimensional coordinates of the local coordinate system of the marker points on each image after absolute orientation to solve the extrinsic parameter values ​​of each calibration image relative to the world coordinate system; Optimize and adjust the camera parameters: use the collinear equation as the observation equation, perform full parameter bundling adjustment, and solve and output the internal and external parameter values ​​of the camera system with the constraint that the internal parameters of each camera and the relative external parameters of the non-reference camera do not change; Before absolutely orienting each camera, the following is also included: Correct the initial value of relative extrinsic parameters by using the scale; take the local coordinate axis vector as the unit vector, introduce the scale factor, and use the geometric relationship that the sum of the distances from the marker point to its center of mass remains unchanged to establish the corresponding relationship between the local coordinate distance and the actual physical distance, and solve to obtain the initial value of the relative extrinsic parameters of the non-reference camera; Absolutely orient each camera and solve the extrinsic parameters of the calibration image acquired by each camera, including: Determine the center of mass position; According to the set of three-dimensional coordinate points {p} determined in any image, the three-dimensional coordinates of any point p n (x n ,y n , z n ), then the center of mass M p Location is Calculate the distance from each point to the centroid; According to the three-dimensional coordinate point set {p} and the centroid M p The position coordinates of each point are used to determine the distance d from the center of mass. n And the sum of the distances from all points to the centroid is: D n =d1+d2+…+d n ; Determine the proportional relationship between the images; Combined with the sum of the distances D from each group of landmarks to the centroid in the world coordinate system, the proportional relationship between the image space auxiliary coordinate system of the left image and the world coordinate system is determined: The proportional relationship S n Applied to the three-dimensional coordinate point set {p}, the corrected left image space is the same size as the world coordinate system, so there exists a set of rotation and translation matrix relationships that satisfy: P=S n *p; P=R -1 (Q-T); Among them, R represents the rotation matrix relationship from the world coordinate system to the image space auxiliary coordinate system, and T represents the translation matrix relationship from the world coordinate system to the image space auxiliary coordinate system; The two coordinate sets satisfy the SVD singular value decomposition method. Using the SVD singular value decomposition method, we can solve R and T and determine the external parameters of each image relative to the world coordinate system.

2. A single scale on-site calibration method according to claim 1, characterized in that: Using a ruler and a binocular camera to obtain calibration images includes: Step 1.1, designing marking points: using dot matrix coding points as marking points, the marking points are composed of 8 dots with the same diameter, among which the points at the five positions of A, B, C, D, and E are used as template points, and the coding area is located by using the cross ratio invariance principle. The other three marking points are assigned unique digital numbers according to their different positions in the design coordinate system, which are used to distinguish the overall dot matrix coding; According to perspective geometry, the projection of the image target from the three-dimensional world coordinate system to the two-dimensional measurement coordinate system of the image follows the nonlinear perspective projection transformation; the cross ratio invariant is the invariant under perspective projection, that is: Among them, A, B, and C are collinear to obtain a straight line BC, and the intersection of the straight line BC and the line connecting E and D is F, and C r =1.5112; A', B', C', D', E', F' are the projection points of A, B, C, D, E, F respectively; Step 1.2, marker detection: Step 1.2.1, ellipse recognition: perform noise reduction, sub-pixel edge extraction and ellipse fitting on the acquired image in sequence to obtain the coordinates of the ellipse center point and the major semi-axis length parameters; Step 1.2.2, clustering to generate point clusters: randomly divide the obtained ellipse center points into multiple point clusters, and the number of points in each point cluster is not less than 8; Step 1.2.3, identify template points: set the point closest to the center of gravity of the point cluster as template point A; set the point farthest from template point A as template point B; template point C is collinear with points AB and is located on the reverse extension line of the AB line; D and E are located on both sides of point C and the angle between them and the two vectors is not less than 20°; Step 1.2.4, calculate the cross ratio: identify the five template points and substitute them into the cross ratio formula, and output the corresponding cross ratio value; if the cross ratio value is 1.5112, it is dot matrix coding, and go to the next step; otherwise, return to re-identify template point B; Step 1.2.5, calculate the affine transformation parameters: according to the two-dimensional image coordinates of the five template points and the given two-dimensional design coordinates, solve the affine transformation parameters between the two coordinate systems; Step 1.2.6, marking point matching: according to the obtained affine transformation parameters and the image coordinates of the other three points except the template point, the design coordinates corresponding to the three points are obtained; Step 1.2.7, solve the final code: according to the coding principle, design the coordinates to find the corresponding coding ID, and output the final ID of all marked points in the point block; Step 1.2.8, image calibration: use a straight ruler to collect the calibration image of the binocular camera.

3. A single scale on-site calibration method according to claim 1, characterized in that: Relative orientation of each camera, including: The coplanarity condition equation is satisfied: The first row represents the three components of the photographic baseline relative to the image space auxiliary coordinate system (B x , B y , B z ), that is, the coordinates of the origin of the right image coordinate system in the auxiliary coordinate system of the image space; The second line indicates the coordinates of the left image point in the left coordinate system (X1, Y1, Z1); The third line represents the coordinates of the right image point in the auxiliary coordinate system of the image space (X2+B x ,Y2+B y ,Z2+B z ); According to the properties of the determinant, we can get: The measurement results of the left and right image point coordinates are expressed as follows: Where f1 represents the principal point deviation of the left camera; f2 represents the principal point deviation of the right camera; ω、 κ represents the angle components of the principal point position of the right camera relative to the principal point position of the left camera in the X, Y, and Z directions in the auxiliary coordinate system of the image space; B x , B y , B z They respectively represent the components of the photographic baseline formed by the line connecting the principal points of the left and right cameras in the X, Y, and Z directions in the image space coordinate system; R represents the rotation matrix of the right image point from its own camera coordinate system to the auxiliary coordinate system of the image space; B y , B z It is expressed as follows: Finally, the coplanar condition equation is transformed into a nonlinear equation, namely: Where μ represents the X-axis component B of the auxiliary coordinate system of the photographic baseline image space x and the Y-axis component B of the auxiliary coordinate system of the photographic baseline image space y The complementary angle component of the angle between ; ν represents the photographic baseline B and the Z-axis component B of the photographic baseline in the auxiliary coordinate system of the image space z The complementary angle of the angle; According to the rear intersection method, if we want to get the relative orientation result, we first perform Taylor expansion on the coplanar condition equation, that is: Partial derivatives solution: Dividing both sides by (X2Z1-X1Z2) yields: At the same time, the parallax W is introduced: W = N1Y1-(N2Y2+B y ); in, but: because, but: The parallax W error equation is obtained: Among them, each pair of homologous image points has a set of parallax W. More than 7 pairs of homologous image points are measured and the parallax is set to zero to complete the relative orientation of the camera. The relative orientation parameters and the initial values ​​of the intrinsic parameters are solved according to the least squares adjustment method.

4. A single scale on-site calibration method according to claim 3, characterized in that: Solving the relative orientation parameters and the initial values ​​of the internal parameters using the least squares adjustment method includes: Obtain known image data: determine the camera's factory focal length parameters and the left image point coordinates (x1, y1) and the right image point coordinates (x2, y2); Assumed photographic baseline: Assume the photographic baseline is B x =x1-x2; Set all parameters to zero: Assume that the initial values ​​of all relative orientation parameters are zero; the principal point distance between the two cameras is f1 = f2 = 0; Solve image space auxiliary coordinates: calculate image space auxiliary coordinates (X1, Y1, Z1) and (X2, Y2, Z2); solve the error equation: calculate the coefficients and constant terms of the error equation point by point and convert them into equations; Iteratively determine the initial value of the element: According to the same-plane constraint relationship, the parallax equation is required to be zero, and the solution is continuously iterated. After the final iteration converges, the final relative orientation parameter value and the intrinsic parameter values ​​of the two cameras are output.

5. A single scale on-site calibration method according to claim 1, characterized in that: Correcting the initial values ​​of relative external parameters by scale includes: Determine the coordinates (X) of the jth group of landmarks in the i-th right image in the image space auxiliary coordinate system. 2j , Y 2j , Z 2j ), we can see that the centroid coordinates in this image can be expressed as: Find the distance from the centroid to the jth set of landmarks in the i-th right image and sum the distances from all points in the image to the centroid: d i =d i1 +d i2 +…+d ij ; Among them, d ij represents the distance from the jth group of landmark points to the centroid in the i-th image; i represents the number of images captured, i=1, 2, 3, ...; j represents the number of landmark point groups captured in the i-th image, j=1, 2, 3, ...; d i Represents the sum of the distances from each group of landmark points to the centroid in the i-th image; The sum of the distances from the markers to the centroid on each image remains unchanged: In the formula, It represents the mean of the sum of the distances from each group of landmark points to the centroid in all images taken in the image space auxiliary coordinate system, where d1+d2+…+d i Its sum is constant and remains unchanged; According to the DP close-range industrial photogrammetry system, the global point coordinate set {N} of all the marking points on the ruler is obtained. Assume that the coordinate of one of the points in the object space is (X j , Y j , Z j ), solve the centroid coordinates of the global point and find the sum of the distances from each point to the centroid; According to the global point coordinates, the centroid coordinates are obtained: Solve the distance D from each group of landmark points to the centroid in the image j : And the distance D from each group of landmark points to the centroid j Summing, we get D: D=D1+D2+…+D j ; Determine the distance ratio relationship between the two groups as follows: Where: S represents the sum of the distances from each group of landmarks to the centroid in the world coordinate system and the mean of the sum of the distances from each group of landmarks to the centroid in all images captured in the image space auxiliary coordinate system. The ratio of the image space auxiliary coordinate system to the world coordinate system, and the proportional relationship is applied to the three components of the photographic baseline relative to the image space auxiliary coordinate system (B x , B y , B z ), combined with the three obtained angle elements (ω, φ, κ), determine the initial values ​​of the relative external parameters of the current camera system.

6. A single scale on-site calibration method according to claim 1, characterized in that: Using the SVD singular value decomposition method, solving the matrix transformation relationship between the two coordinate sets includes: Solve for the matrix centroid; Assume that the centroids of matrices Q and P are but: Translation matrix; Translate the matrices Q and P relative to their respective centroid positions, and the new matrices are Q′ and P′ respectively, then: SVD decomposition: Use matrices Q and P to construct matrix M, and perform SVD decomposition on it: Solve R and T; 7. A single scale on-site calibration method according to claim 1, characterized in that: Optimize and adjust the camera parameters, including: Use the error equation as the observation equation for all images of the left and right cameras respectively; For the benchmark camera: V L =A L X 1L +B L X 2L +C L X 3L -L; Where: X 1L , X 2L , X 3L are the internal parameters, external parameters and correction numbers of the three-dimensional coordinates of the reference camera; A L , B L , C L are the partial derivative matrices corresponding to the intrinsic parameters, extrinsic parameters and object point coordinates of the reference camera respectively; L is the deviation between the observed true value and the initial value; Also for non-reference cameras, there is an error equation: V R =A R X 1R +B R X 2L +C R X 3R +D R X 4R -L; Among them: A R represents the partial derivative matrix of the intrinsic parameters of the non-reference camera; X 1R represents the intrinsic parameters of the non-reference camera; X 3R Represents the three-dimensional coordinates of the object point; B R D represents the partial derivative matrix of the image point coordinates of the non-reference camera relative to the reference camera extrinsic parameters; R represents the partial derivative matrix of the image point of the non-reference camera with respect to its external parameters; X 4R Indicates the correction number of the relative external parameters; C R Represent the partial derivative matrix corresponding to the object point coordinates of the non-reference camera; determine the global point coordinates of the marker point, that is, the world coordinates of the marker point; When the intrinsic parameters of the camera and the relative extrinsic parameters of the non-reference camera do not change, the final intrinsic and extrinsic parameters of the camera and the relative extrinsic parameter values ​​of the non-reference camera are output, and the intrinsic and extrinsic parameter calibration of the camera system is completed.

8. A single scale on-site calibration method according to claim 2, characterized in that: The straight ruler is a carbon fiber straight ruler; The marking points adopt a structure of a metal base film and reflective micro-bead dots, and the reflectivity of each marking point is 300 mesh to 400 mesh; When the one-line ruler is used in the horizontal and vertical directions, the one-line ruler moves up, down, left, and right in a fixed plane; When calibrating the depth of field direction, the front depth of field and the back depth of field positions are photographed simultaneously, and the marking point of the one-line ruler in the depth of field direction is inclined at an angle of 45° with the ruler body of the one-line ruler.

Images