Line scan camera calibration method based on auxiliary camera and related apparatus

Abstract

The present application discloses a line scan camera calibration method based on an auxiliary camera and a related apparatus. The method comprises: calibrating an auxiliary camera, rigidly combining the calibrated auxiliary camera and a line scan camera to be calibrated and mounting same on a motion platform, and placing a checkerboard calibration plate in an imaging area of a combined camera; driving the motion platform to obtain calibration plate images captured by the auxiliary camera at an initial position and an ending position and a dynamic scanning image of the line scan camera, and calculating a motion vector of the combined camera under an auxiliary camera coordinate system; calculating the coordinates of registration points under the auxiliary camera coordinate system by means of the motion vector and the dynamic scanning image; fitting an equation of an imaging plane of the line scan camera under the auxiliary camera coordinate system by means of the coordinates of the plurality of registration points under the auxiliary camera coordinate system; and on the basis of the properties of a rotation matrix and a line scan camera imaging model, calculating internal parameters of the line scan camera and a rigid transformation matrix between the auxiliary camera and the line scan camera. Therefore, the calibration precision of line scan cameras is improved.

Description

A line array camera calibration method based on an auxiliary camera and related device TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, and in particular to a line array camera calibration method based on an auxiliary camera and related device. BACKGROUND

[0002] In the field of optical measurement, camera calibration is a key step aimed at determining the internal parameters (such as focal length, optical center position, etc.) and external parameters (such as rotation matrix and translation vector) of the camera to achieve high-precision three-dimensional measurement and positioning. As a common industrial camera, the line array camera has certain particularity in its calibration process due to its structural characteristics. Existing line array camera calibration methods are usually based on one-dimensional calibration objects, and the camera parameters are estimated by acquiring images at different angles. However, these methods are often affected by factors such as calibration object manufacturing precision and image acquisition conditions, making it difficult to achieve high-precision calibration results.

[0003] SUMMARY

[0004] The present application provides a line array camera calibration method based on an auxiliary camera and related device to improve the calibration accuracy of line array cameras.

[0005] Therefore, the first aspect of the present application provides a line array camera calibration method based on an auxiliary camera, comprising:

[0006] Calibrate the auxiliary camera, rigidly combine the calibrated auxiliary camera with the line array camera to be calibrated, and install them on a motion platform. Place a checkerboard calibration board in the imaging area of the combined camera.

[0007] Acquire calibration board images taken by the auxiliary camera at the initial position and the end position, and dynamic scanning images of the line array camera by driving the motion platform, and calculate the motion vector of the combined camera in the auxiliary camera coordinate system.

[0008] Calculate the coordinates of the registration points in the auxiliary camera coordinate system through the motion vector and the dynamic scanning images.

[0009] Fit the equation of the line array camera imaging plane in the auxiliary camera coordinate system through the coordinates of multiple registration points in the auxiliary camera coordinate system.

[0010] Calculate the internal parameters of the line array camera and the rigid transformation matrix between the auxiliary camera and the line array camera based on the properties of the rotation matrix and the line array camera imaging model.

[0011] Optionally, the method further comprises:

[0012] acquiring the image of the calibration board captured by the auxiliary camera at the initial position, driving the motion platform to make the combined camera move along the straight line by the preset distance, making the line array camera acquire the dynamic scanning image of the target number of rows, and acquiring the image of the calibration board captured by the auxiliary camera at the end position of the motion;

[0013] calculating the extrinsic parameters of the auxiliary camera at the initial position and the extrinsic parameters of the auxiliary camera at the end position based on the images of the calibration board captured by the auxiliary camera at the initial position and the end position, and using the N-point perspective monocular vision positioning method;

[0014] calculating the motion vector of the combined camera in the coordinate system of the auxiliary camera according to the extrinsic parameters of the auxiliary camera at the initial position, the extrinsic parameters of the auxiliary camera at the end position, and the target number of rows.

[0015] Optionally, the method further comprises:

[0016] calculating the coordinates of the registration point in the coordinate system of the auxiliary camera by using the motion vector, the sub-pixel coordinates of the registration point in the dynamic scanning image, and the sub-pixel coordinates of the corner points of the checkerboard in the dynamic scanning image.

[0017] Optionally, the method further comprises:

[0018] transforming the coordinates of the registration point in the coordinate system of the auxiliary camera to the coordinate system of the line array camera, and then transforming the coordinates of the registration point in the coordinate system of the line array camera to the pixel coordinate system to obtain an intermediate relationship;

[0019] separating the known parameters from the intermediate relationship to obtain an unknown parameter relationship;

[0020] acquiring the unit normal vector of the imaging plane of the line array camera according to the equation of the imaging plane of the line array camera in the coordinate system of the auxiliary camera, and solving the internal parameters of the line array camera and the rigid transformation matrix between the auxiliary camera and the line array camera based on the unit normal vector and the unknown parameter relationship.

[0021] Optionally, the method further comprises:

[0022] establishing a camera parameter optimization model according to the lens distortion model of the line array camera and the least square optimization principle;

[0023] by solving the camera parameter optimization model, obtaining the parameter correction of the linear array camera;

[0024] by the parameter correction, correcting the internal parameter of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera, obtaining the optimized internal parameter of the linear array camera and the optimized rigid transformation matrix.

[0025] The second aspect of the application provides a linear array camera calibration device based on an auxiliary camera, comprising:

[0026] a combination unit configured to calibrate the auxiliary camera, rigidly combine the calibrated auxiliary camera with a linear array camera to be calibrated, and install the combination on a motion platform, and place a checkerboard calibration board in the imaging area of the combination camera;

[0027] a first calculation unit configured to acquire calibration board images captured by the auxiliary camera at initial and end positions and dynamic scanning images of the linear array camera by driving the motion platform, and calculate a motion vector of the combination camera in the auxiliary camera coordinate system;

[0028] a second calculation unit configured to calculate coordinates of registration points in the auxiliary camera coordinate system by using the motion vector and the dynamic scanning images;

[0029] a fitting unit configured to fit an equation of an imaging plane of the linear array camera in the auxiliary camera coordinate system by using the coordinates of the registration points in the auxiliary camera coordinate system;

[0030] a third calculation unit configured to calculate the internal parameter of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera based on the properties of the rotation matrix and the imaging model of the linear array camera.

[0031] Optionally, the first calculation unit is specifically configured to:

[0032] acquire the calibration board image captured by the auxiliary camera at the initial position, drive the motion platform to make the combination camera move along a straight line by a preset distance, make the linear array camera acquire dynamic scanning images of a target number of rows, and acquire the calibration board image captured by the auxiliary camera at the end position of the motion;

[0033] based on the calibration board images captured by the auxiliary camera at the initial and end positions, calculate the external parameter of the auxiliary camera at the initial position and the external parameter of the auxiliary camera at the end position by using an N-point perspective monocular vision positioning method;

[0034] calculate the motion vector of the combination camera in the auxiliary camera coordinate system according to the external parameter of the auxiliary camera at the initial position, the external parameter of the auxiliary camera at the end position, and the target number of rows.

[0035] Optionally, the second calculation unit is specifically configured to:

[0036] The coordinates of the registration points in the auxiliary camera coordinate system are calculated by the motion vector, the sub-pixel coordinates of the registration points in the dynamic scanning image, and the sub-pixel coordinates of the chessboard corner points in the dynamic scanning image.

[0037] The third aspect of the present application provides an electronic device, the device comprising a processor and a memory;

[0038] The memory is configured to store program code and transmit the program code to the processor;

[0039] The processor is configured to execute the auxiliary camera-based linear array camera calibration method according to the instructions in the program code.

[0040] The fourth aspect of the present application provides a computer readable storage medium for storing program code, the program code being executed by a processor to implement the auxiliary camera-based linear array camera calibration method.

[0041] From the above technical solutions, the present application has the following advantages:

[0042] The present application provides an auxiliary camera-based linear array camera calibration method, comprising: calibrating an auxiliary camera, rigidly combining the calibrated auxiliary camera with a linear array camera to be calibrated, and mounting the combined cameras on a motion platform, and placing a chessboard calibration board in the imaging area of the combined cameras; acquiring calibration board images captured by the auxiliary camera at initial and end positions, and dynamic scanning images of the linear array camera by driving the motion platform, and calculating a motion vector of the combined cameras in the auxiliary camera coordinate system; calculating coordinates of registration points in the auxiliary camera coordinate system by the motion vector and the dynamic scanning images; fitting an equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system by the coordinates of the registration points in the auxiliary camera coordinate system; and calculating the intrinsic parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera based on the properties of the rotation matrix and the imaging model of the linear array camera.

[0043] In the present application, the calibrated auxiliary camera is rigidly combined with the linear array camera to be calibrated, and high-precision calibration is performed on the motion platform, the linear array camera is calibrated based on the auxiliary camera, which not only reduces the number of images required during calibration, but also improves the calibration accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0045] Fig. 1 is a flow diagram of a line array camera calibration method based on an auxiliary camera according to an embodiment of the present application;

[0046] Fig. 2 is a schematic diagram of obtaining a registration point coordinate according to an embodiment of the present application;

[0047] Fig. 3 is a structural schematic diagram of a line array camera calibration device based on an auxiliary camera according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0049] For the convenience of understanding, please refer to Fig. 1. The present application provides a line array camera calibration method based on an auxiliary camera, comprising:

[0050] Step 101, calibrate the auxiliary camera, rigidly combine the calibrated auxiliary camera with the line array camera to be calibrated, and install them on a motion platform. Place the checkerboard calibration board in the imaging area of the combined camera.

[0051] A face array camera can be used as an auxiliary camera, and then Zhang Zhengyou calibration method can be used to calibrate the auxiliary camera to obtain the intrinsic parameters P of the auxiliary camera f .

[0052] Rigidly connect the calibrated auxiliary camera with the line array camera to be calibrated to obtain a combined camera, and install the combined camera on a single-axis high-precision linear motion platform. Then, place the checkerboard calibration board in the imaging area of the combined camera so that the auxiliary camera and the line array camera can both shoot the checkerboard calibration board.

[0053] Step 102, obtain the calibration board images shot by the auxiliary camera at the initial position and the end position and the dynamic scanning image of the line array camera by driving the motion platform, and calculate the motion vector of the combined camera in the auxiliary camera coordinate system.

[0054] The image of the calibration board taken by the auxiliary camera at the initial position is acquired, the motion platform is driven to make the combined camera move along a straight line by a preset distance, the dynamic scanning image of the target number of rows is acquired by the linear array camera, and the image of the calibration board taken by the auxiliary camera at the end position of the motion is acquired; based on the images of the calibration board taken by the auxiliary camera at the initial position and the end position, the N-point perspective monocular visual positioning method is used to calculate the extrinsic parameters of the auxiliary camera at the initial position and the extrinsic parameters of the auxiliary camera at the end position; and the motion vector of the combined camera in the auxiliary camera coordinate system is calculated according to the extrinsic parameters of the auxiliary camera at the initial position, the extrinsic parameters of the auxiliary camera at the end position and the target number of rows.

[0055] After the chessboard calibration board is placed, the image of the chessboard is acquired by the auxiliary camera to obtain the image of the chessboard at the initial position, then the motion platform is driven to make the combined camera move along a straight line by a preset distance, the dynamic scanning image of the target number of rows is acquired by the linear array camera, and the image of the chessboard at the end position of the motion is acquired by the auxiliary camera. The N-point perspective monocular visual positioning method is applied to obtain the extrinsic parameters R f (0) of the auxiliary camera at the initial position and the extrinsic parameters R f (0) of the auxiliary camera at the end position. f (n) f (n) The motion vector V f of the combined camera in the auxiliary camera coordinate system is obtained by the following formula (unit: mm / pixel):

[0056] In the formula, V f-X , V f-Y and V f-z are the motion velocities of the combined camera in the X, Y and Z directions in the auxiliary camera coordinate system.

[0057] Step 103, the coordinates of the registration points in the auxiliary camera coordinate system are calculated by the motion vector and the dynamic scanning image.

[0058] The coordinates of the registration points in the auxiliary camera coordinate system are calculated by the motion vector, the sub-pixel coordinates of the registration points in the dynamic scanning image and the sub-pixel coordinates of the corner points of the chessboard in the dynamic scanning image. Specifically, first, the sub-pixel corner point coordinates of the dynamic scanning image of the linear array camera are extracted, and the sub-pixel coordinates [u P0 , v P0 ], [u P1 , v P1 ] and [u P2 , v P2 of multiple corner points such as 0, 1, 2, … are obtained.As shown in FIG. 2). Then, the relationship between the corner point coordinates and the registration point coordinates can be obtained by using the non-perspective and non-distortion characteristics of the linear array camera scanning direction:

[0059] In the formula, X WP5 is the X coordinate of the corner point 5 in the world coordinate system, X WP6 is the X coordinate of the corner point 6 in the world coordinate system, and X WIS3 is the X coordinate of the registration point 3 in the world coordinate system. The registration point is the intersection point of the imaging plane of the linear array camera and the grid line of the checkerboard calibration plate, and can be used to establish the corresponding relationship between different coordinate systems, thereby realizing the conversion or transformation of the coordinate systems.

[0060] The extrinsic parameters R f (0) , T f (0) and the motion vector V f of the auxiliary camera in the initial position are obtained. W W W Then, the world coordinates (X f (v) , Y f (v) , Z f (v) ) of the registration point with the y coordinate v in the pixel coordinate system can be transformed into the coordinates (X fl f , Y fl f , Z fl f ) in the auxiliary camera coordinate system by using formula (3):

[0061] Through the above process, the coordinates of the registration point in the auxiliary camera coordinate system can be calculated.

[0062] Step 104: Fitting the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system through the coordinates of the registration points in the auxiliary camera coordinate system.

[0063] By scanning the checkerboard calibration plate in multiple poses, a large number of registration point coordinates in the auxiliary camera coordinate system can be obtained. After excluding the abnormal points in the registration points, the least squares method can be used for plane fitting to obtain the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system: fl X f +B fl Y f +C fl Z f +D fl = 0 (4)

[0064] In the formula, A fl , B fl , C fl , and Dfl are coefficients of the imaging plane equation.

[0065] Step 105, calculating the internal parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera based on the properties of the rotation matrix and the linear array camera imaging model.

[0066] The coordinates of the registration points in the auxiliary camera coordinate system are transformed to the linear array camera coordinate system by using the properties of the rotation matrix in the rigid transformation and the linear array camera imaging model, and then the coordinates of the registration points in the linear array camera coordinate system are transformed to the pixel coordinate system, to obtain an intermediate relationship;

[0067] The known parameters are separated from the intermediate relationship to obtain an unknown parameter relationship;

[0068] The unit normal vector of the linear array camera imaging plane is obtained according to the equation of the linear array camera imaging plane in the auxiliary camera coordinate system, and the internal parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera are solved based on the unit normal vector and the unknown parameter relationship.

[0069] Since the two cameras are rigidly connected, the points in the auxiliary camera coordinate system can be transformed to the linear array camera coordinate system by formula (5). Without considering the lens distortion, the points in the linear array camera coordinate system can be transformed to the pixel coordinate system by the linear array camera imaging model (formula (6)).

[0070] In the formula, R fl represents the rotation matrix in the rigid transformation, T fl represents the translation vector in the rigid transformation, f x represents the ratio of the image distance to the x-direction pixel size, f y represents the ratio of the image distance to the y-direction pixel size, and u0 represents the position of the principal point in the x-direction. The principal point is the point where the imaging plane and the optical axis are perpendicular.

[0071] By using the properties of the rotation matrix, formula (5) and formula (6) can be simplified and combined to obtain formula (7):

[0072] In the formula, r is an element in the rotation matrix R fl , t x , t y , and t z are the translation distances in the x, y, and z directions of the translation vector T fl ,

[0073] According to the direct linear transformation theory, all known parameters are separated (i.e. Xf , Y f , uX f , uY f , u), and recombine all unknown parameters into new coefficients m i (i = 1…6) to get the unknown parameter relationship as shown in equation (8), which can be solved from the over-determined equation group by using the least square method.

[0074] wherein,

[0075] Equation (8) describes the linear array camera imaging model in an ideal case, but for strict geometric calibration, after solving the linear coefficients in the model, further calculation of geometric imaging model parameters with physical meaning from the linear coefficients is needed.

[0076] In the above steps, the equation of the linear array camera imaging plane in the auxiliary camera coordinate system has been obtained, and further the unit normal vector (a fl ,b fl ,c fl ) of the linear array camera imaging plane can be obtained as:

[0077] After the rigid transformation of the rotation matrix R fl , the following can be obtained:

[0078] In the above steps, m4 and m5 have been solved, and m4 = r 12 and m5 = -r 11 are known, according to equation (9), r 13 can be solved, and combined with the properties of the rotation matrix, r 31 can be solved: O = a 2 r 12 4 + a 2 r 12 2 r 13 2 + 2abr 11 r 12 r 13 2 Q = 2acr 11 r 12 2 r 13 + 2ar 11 r 12 3 + 2ar 11 r 12 r 13 2S = b 2 r 11 2 r 13 2 +b 2 r 12 2 r 13 2 W = 2bcr 11 2 r 12 r 13 +2bcr 12 3 r 13 +2br 11 2 r 13 2 E = c 2 r 11 2 r 12 2 +c 2 r 12 4 +2cr 11 2 r 12 r 13 +r 11 2 r 12 2 +r 11 2 r 13 2

[0079] where O, Q, S, W, E are intermediate parameters, a = a fl , b = b fl , c = c fl .

[0080] Assume r 31 > 0, according to the properties of the rotation matrix, r 21 , r 22 , r 23 can be solved. Since the auxiliary camera coordinate system and the linear array camera coordinate system are both right-handed coordinate systems, there is no reflection case, so the value of the determinant of the rotation matrix should be +1. If the determinant of the rotation matrix R fl calculated satisfies the following inequality:

[0081] then the value of the rotation matrix is correct, otherwise, r 31 <0 should be assumed again, and the value of the rotation matrix R fl is calculated.

[0082] m1, m2, R fl , m1 = -u0r 12 -f x r 32 , m2 = u0r 11 +f x r 31 , u0, f x .

[0083] Since the origin of the linear array camera coordinate system is on the imaging plane, the translation vector T fl in the rigid transformation matrix satisfies: A fl t x +B fl t y +C fl t z +D fl = 0 (10)

[0084] Combined with m3 and m6 obtained in the previous steps, the value of the translation vector T fl can be obtained as follows:

[0085] where A = A fl , B = B fl , C = C fl , and D = D fl .

[0086] In the previous steps, the rotation matrix R fl of the rigid transformation between the two cameras has been obtained. The motion vector in the auxiliary camera coordinate system can be converted to the linear array camera coordinate system using formula (11):

[0087] At this point, all 11 parameters in the basic model of the linear array camera have been obtained using linear and analytical calculation methods.

[0088] Further, after step 105, the following can also be included:

[0089] Step 106, optimizing the internal parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera.

[0090] For high-precision camera geometry calibration, the camera parameters obtained above are not the optimal results. The main reason is that the distortion of the linear array camera lens is not considered in the above solving process.

[0091] According to Brown's distortion model theory, the formula for the effect of the distortion parameters of the linear array camera lens on the image point coordinates is: x' = x (1 + k1x 2+k2x 4 )+p1(x 2 +2x 2 )=x+3p1x 2 +k1x 3 +k2x 5 (12)

[0092] In the formula, k1 and k2 represent the radial distortion coefficients of the lens, and p1 represents the tangential distortion coefficient of the lens, which can more accurately model the radial and tangential distortion of the lens.

[0093] According to the lens distortion model of the linear array camera (i.e., formula (12)) and the least square optimization principle, a camera parameter optimization model is established; by solving the camera parameter optimization model, a parameter correction amount of the linear array camera is obtained; by correcting the intrinsic parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera through the parameter correction amount, the optimized intrinsic parameters of the linear array camera and the optimized rigid transformation matrix are obtained.

[0094] The angle of rotation around the Z axis is defined as γ, the angle of rotation around the Y axis is defined as β, and the angle of rotation around the X axis is defined as α. Meanwhile, the rotation component satisfies four conditions: the direction of counterclockwise rotation is the positive direction of rotation, the rotation axis characteristic is rotation around a fixed axis, the angle system is γ-β-α, and the value range of the rotation component is -Π / 2≤β≤Π / 2, -Π≤α≤Π, and -Π≤γ≤Π. Moreover, the world coordinate system, the auxiliary camera coordinate system, and the linear array camera coordinate system are all right-handed coordinate systems, so the rotation matrix R in the rigid transformation can be uniquely decomposed into a set of rotation components: fl

[0095] In visual measurement, the least square solution is usually regarded as the best output result, so the optimal parameter value can be obtained by minimizing the following function:

[0096] In the formula, u i is the X coordinate of the sub-pixel point i detected in the real scan image, is the coordinate obtained by re-projecting the point i using the spatial point and the camera model, and N is the number of re-projected points.

[0097] According to the linear array camera imaging model and the lens distortion model, the formula of the re-projected point coordinate u can be derived as follows:

[0098] where S x is the size of the linear array camera pixel in the X direction, f=f x S x , ​is the re-projection point coordinate of the spatial point in the image coordinate system of the linear array camera without considering distortion x-coordinate value of the re-projection point of the spatial point in the image coordinate system of the linear array camera

[0099] After linearization by first-order Taylor expansion, the error equation is as follows:

[0100] In the formula, (e u ,e v ) is the re-projection error of each point, and Δ represents the correction value of the corresponding parameter.

[0101] Rewrite formula (16) in the form of vector and matrix to obtain the camera parameter optimization model as follows: e i =D i ·M+g i e i =[e u e v ] T M=[Δu0 Δp1 Δk1 Δk2 Δf Δα Δβ Δγ Δt x Δt y Δt z ] T

[0102] Each pair of registration points (u i ,0) and can establish the above formula. According to the least square adjustment theory, let the error vector E=0, and obtain: 0=D·M+G

[0103] Solving the above linear overdetermined equation set can obtain the least square solution M, which is the correction amount of the internal parameter of the linear array camera and the rigid transformation matrix. Through repeated iteration of the above optimization process until the re-projection error meets the needs, the optimal solution M is output, the parameter correction amount of the linear array camera is obtained, the internal parameter of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera are corrected through the parameter correction amount, the optimized internal parameter of the linear array camera and the optimized rigid transformation matrix are obtained, and high-precision calibration of the linear array camera is realized.

[0104] The embodiment of the present application provides a line array camera calibration method based on an auxiliary camera, which uses a plane array camera as an auxiliary device, calibrates a line array camera to be calibrated in a high-precision manner on a motion platform, improves the accuracy and reliability of calibration, is simple to operate, low in cost, and can be widely applied to the industrial measurement field. Compared with the dynamic scanning calibration method commonly used in the industry, the method calibrates the line array camera parameters based on the auxiliary camera, reduces the number of images required during calibration, and improves the calibration accuracy. Compared with the existing joint calibration method of the line array camera and the plane array camera, the method does not need to use a calibration board and a customized irregular image plate simultaneously. Compared with the existing static calibration method, the method adopts a dynamic scanning mode for calibration, a large number of registration points can be obtained, and the calibration precision is improved.

[0105] Further, the present application considers the distortion of the line array camera lens, establishes a camera parameter optimization model according to a line array camera lens distortion model and a least square optimization principle, obtains line array camera optimization parameters by optimizing the camera parameter optimization model, and further optimizes the calibration precision of the line array camera.

[0106] The above is one embodiment of the line array camera calibration method based on the auxiliary camera provided by the present application, and the following is a line array camera calibration device based on the auxiliary camera provided by the present application.

[0107] Referring to FIG. 3, the line array camera calibration device based on the auxiliary camera provided by the embodiment of the present application comprises:

[0108] The combination unit is configured to calibrate the auxiliary camera, rigidly combine the calibrated auxiliary camera with the line array camera to be calibrated, and install the combination camera on the motion platform, and place the checkerboard calibration board in the imaging area of the combination camera.

[0109] The first calculation unit is configured to acquire the calibration board images and the dynamic scanning images of the line array camera by driving the motion platform to capture the calibration board images and the dynamic scanning images of the line array camera at the initial position and the end position of the auxiliary camera, and calculate the motion vector of the combination camera in the auxiliary camera coordinate system.

[0110] The second calculation unit is configured to calculate the coordinates of the registration points in the auxiliary camera coordinate system by using the motion vector and the dynamic scanning images.

[0111] The fitting unit is configured to fit the equation of the line array camera imaging plane in the auxiliary camera coordinate system by using the coordinates of the plurality of registration points in the auxiliary camera coordinate system.

[0112] The third calculation unit is configured to calculate the internal parameters of the line array camera and the rigid transformation matrix between the auxiliary camera and the line array camera based on the properties of the rotation matrix and the line array camera imaging model.

[0113] As a further improvement, the first computing unit is specifically configured to:

[0114] acquire the calibration board image captured by the auxiliary camera at the initial position, drive the motion platform to make the combined camera move along the straight line by the preset distance, make the linear array camera acquire the dynamic scanning image of the target number of rows, and acquire the calibration board image captured by the auxiliary camera at the end position of the motion;

[0115] based on the calibration board images captured by the auxiliary camera at the initial position and the end position, calculate the extrinsic parameters of the auxiliary camera at the initial position and the extrinsic parameters of the auxiliary camera at the end position by using an N-point perspective monocular vision positioning method;

[0116] calculate the motion vector of the combined camera in the auxiliary camera coordinate system according to the extrinsic parameters of the auxiliary camera at the initial position, the extrinsic parameters of the auxiliary camera at the end position, and the target number of rows.

[0117] As a further improvement, the second computing unit is specifically configured to:

[0118] calculate the coordinates of the registration points in the auxiliary camera coordinate system by using the motion vector, the sub-pixel coordinates of the registration points in the dynamic scanning image, and the sub-pixel coordinates of the chessboard corner points in the dynamic scanning image.

[0119] As a further improvement, the third computing unit is specifically configured to:

[0120] transform the coordinates of the registration points in the auxiliary camera coordinate system to the linear array camera coordinate system by using the properties of the rotation matrix in the rigid transformation and the imaging model of the linear array camera, and then transform the coordinates of the registration points in the linear array camera coordinate system to the pixel coordinate system to obtain an intermediate relationship;

[0121] separate the known parameters from the intermediate relationship to obtain an unknown parameter relationship;

[0122] acquire the unit normal vector of the imaging plane of the linear array camera according to the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system, and solve the internal parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera based on the unit normal vector and the unknown parameter relationship.

[0123] As a further improvement, the device further comprises an optimization unit configured to:

[0124] establish a camera parameter optimization model according to the lens distortion model of the linear array camera and the least squares optimization principle;

[0125] obtain the parameter correction amount of the linear array camera by solving the camera parameter optimization model;

[0126] The internal parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera are corrected by the parameter correction amount, to obtain the optimized internal parameters of the linear array camera and the optimized rigid transformation matrix.

[0127] The embodiment of the application calibrates the linear array camera on the motion platform with high precision by rigid combination, which improves the accuracy and reliability of calibration, is simple and low in cost, and can be widely applied in industrial measurement. Compared with the dynamic scanning calibration method commonly used in the industry, the method calibrates the linear array camera based on the auxiliary camera, reduces the number of images required during calibration, and improves the calibration accuracy. Compared with the existing joint calibration method of linear array camera and area array camera, the method does not need to use a calibration board and a customized irregular image panel at the same time. Compared with the existing static calibration method, the method calibrates in a dynamic scanning mode, can obtain a large number of registration points, and improves the calibration accuracy.

[0128] Further, the application considers the distortion of the linear array camera lens, establishes a camera parameter optimization model according to the linear array camera lens distortion model and the least square optimization principle, and obtains the linear array camera optimization parameters by optimizing the camera parameter optimization model, to further optimize the calibration accuracy of the linear array camera.

[0129] The embodiment of the application also provides an electronic device, which comprises a processor and a memory.

[0130] The memory is used to store program code and transmit the program code to the processor.

[0131] The processor is used to execute the linear array camera calibration method based on the auxiliary camera in the foregoing method embodiments according to the instructions in the program code.

[0132] The embodiment of the application also provides a computer readable storage medium, which is used to store program code, and the program code is executed by the processor to execute the linear array camera calibration method based on the auxiliary camera in the foregoing method embodiments.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0134] The terms "first", "second", "third", "fourth", and the like in the description and in the claims of this application, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of accomplishing functionalities that are either the same or similar to that of other embodiments of the application. Moreover, the terms "include", "have", and the like, are used in the detailed description and in the claims of this application essentially open- ended and are intended to encompass the items listed thereafter, equivalents thereof, as well as additional items not listed after the comma. Finally, terms of degree such as "substantially", "approximately", and the like, are used to describe and account for subjective measurements that are inherently imprecise.

[0135] It should be understood that, in this application, "at least one" means one or more, "multiple" means two or more. "And / or", used to describe the relationship between associated objects, means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including single or multiple combinations. For example, at least one of a, b or c, can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0136] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of the units is only a logical functional division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0137] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0138] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0139] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of the present application by a computer device (which can be a personal computer, a server, or a network device, etc.). The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (English full name: Read-Only Memory, English abbreviation: ROM), a random access memory (English full name: Random Access Memory, English abbreviation: RAM), a magnetic disk or an optical disk.

[0140] The above-described and above-mentioned embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A linear array camera calibration method based on an auxiliary camera, characterized in that: include: Calibrate the auxiliary camera by rigidly combining the calibrated auxiliary camera with the line array camera to be calibrated and mounting them on a motion platform. Place a checkerboard calibration plate within the imaging area of ​​the combined camera. The motion platform is driven to obtain the calibration plate images taken by the auxiliary camera at the initial and end positions and the dynamic scanning images of the linear array camera, and the motion vector of the combined camera in the auxiliary camera coordinate system is calculated; Calculating the coordinates of the registration point in the auxiliary camera coordinate system using the motion vector and the dynamic scanning image; Fitting the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system through the coordinates of multiple registration points in the auxiliary camera coordinate system; Based on the properties of the rotation matrix and the linear scan camera imaging model, the intrinsic parameters of the linear scan camera and the rigid transformation matrix between the auxiliary camera and the linear scan camera are calculated.

2. The method for calibrating a linear array camera based on an auxiliary camera according to claim 1, wherein: The method of obtaining the calibration plate image captured by the auxiliary camera at the initial position and the end position and the dynamic scanning image of the line array camera by driving the motion platform, and calculating the motion vector of the combined camera in the auxiliary camera coordinate system, includes: Acquire the calibration plate image taken by the auxiliary camera at the initial position, drive the motion platform to make the combined camera move a preset distance along a straight line, make the line array camera acquire a dynamic scanning image of the target number of rows, and acquire the calibration plate image of the auxiliary camera at the end position of the movement; Based on the calibration plate images taken by the auxiliary camera at the initial position and the final position, the N-point perspective monocular vision positioning method is used to calculate the extrinsic parameters of the auxiliary camera at the initial position and the final position; The motion vector of the combined camera in the auxiliary camera coordinate system is calculated according to the extrinsic parameters of the auxiliary camera at the initial position, the extrinsic parameters of the end position, and the target number of rows.

3. The method for calibrating a linear array camera based on an auxiliary camera according to claim 1, wherein: The calculating the coordinates of the registration point in the auxiliary camera coordinate system by using the motion vector and the dynamic scanning image includes: The coordinates of the registration point in the auxiliary camera coordinate system are calculated using the motion vector, the sub-pixel coordinates of the registration point in the dynamic scanning image, and the sub-pixel coordinates of the checkerboard corner points in the dynamic scanning image.

4. The method for calibrating a linear array camera based on an auxiliary camera according to claim 1, wherein: Based on the properties of the rotation matrix and the linear scan camera imaging model, the intrinsic parameters of the linear scan camera and the rigid transformation matrix between the auxiliary camera and the linear scan camera are calculated, including: By using the properties of the rotation matrix in the rigid transformation and the linear array camera imaging model, the coordinates of the registration points in the auxiliary camera coordinate system are transformed to the linear array camera coordinate system, and then the coordinates of the registration points in the linear array camera coordinate system are transformed to the pixel coordinate system to obtain the intermediate relationship. Separating known parameters from the intermediate relational expression to obtain an unknown parameter relational expression; The unit normal vector of the imaging plane of the linear array camera is obtained according to the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system, and the intrinsic parameters of the linear array camera and the rigid transformation matrix between the auxiliary camera and the linear array camera are obtained based on the relationship between the unit normal vector and the unknown parameters.

5. The method for calibrating a linear array camera based on an auxiliary camera according to claim 4, wherein: The method further comprises: Based on the lens distortion model of the linear array camera and the least squares optimization principle, a camera parameter optimization model is established; Obtaining parameter corrections for the line scan camera by solving the camera parameter optimization model; The intrinsic parameters of the line array camera and the rigid transformation matrix between the auxiliary camera and the line array camera are corrected by the parameter correction amount to obtain the optimized intrinsic parameters of the line array camera and the optimized rigid transformation matrix.

6. A linear array camera calibration device based on an auxiliary camera, characterized in that: include: The combination unit is used to calibrate the auxiliary camera, rigidly combine the calibrated auxiliary camera and the line array camera to be calibrated and install them on the motion platform, and place the checkerboard calibration plate in the imaging area of ​​the combined camera; The first calculation unit is used to obtain the calibration plate image taken by the auxiliary camera at the initial position and the end position and the dynamic scanning image of the line array camera by driving the motion platform, and calculate the motion vector of the combined camera in the auxiliary camera coordinate system; A second calculation unit, configured to calculate the coordinates of the registration point in the auxiliary camera coordinate system using the motion vector and the dynamic scan image; A fitting unit, configured to fit the equation of the imaging plane of the linear array camera in the auxiliary camera coordinate system through the coordinates of the plurality of registration points in the auxiliary camera coordinate system; The third calculation unit is used to calculate the linear array based on the properties of the rotation matrix and the linear array camera imaging model. The intrinsic parameters of the camera and the rigid transformation matrix between the auxiliary camera and the line scan camera.

7. The line array camera calibration device based on an auxiliary camera according to claim 6, characterized in that: The first computing unit is specifically configured to: Acquire the calibration plate image taken by the auxiliary camera at the initial position, drive the motion platform to make the combined camera move a preset distance along a straight line, make the line array camera acquire a dynamic scanning image of the target number of rows, and acquire the calibration plate image of the auxiliary camera at the end position of the movement; Based on the calibration plate images taken by the auxiliary camera at the initial position and the final position, the N-point perspective monocular vision positioning method is used to calculate the extrinsic parameters of the auxiliary camera at the initial position and the final position; The motion vector of the combined camera in the auxiliary camera coordinate system is calculated according to the extrinsic parameters of the auxiliary camera at the initial position, the extrinsic parameters of the end position, and the target number of rows.

8. The line array camera calibration device based on an auxiliary camera according to claim 6, characterized in that: The second computing unit is specifically configured to: The coordinates of the registration point in the auxiliary camera coordinate system are calculated using the motion vector, the sub-pixel coordinates of the registration point in the dynamic scanning image, and the sub-pixel coordinates of the checkerboard corner points in the dynamic scanning image.

9. An electronic device, characterized in that: The device includes a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the auxiliary camera-based line array camera calibration method according to any one of claims 1 to 5 according to instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and when the program code is executed by the processor, the line array camera calibration method based on the auxiliary camera according to any one of claims 1 to 5 is implemented.

Images