A device and method for calibrating camera extrinsic parameters for large separation, no common field of view

Abstract

The application discloses a device and method for calibrating camera external parameters with large spacing and no common field of view, which comprises a translation track and an adjusting seat, the adjusting seat can slide on the translation track, and comprises a lifting rod, a rotating seat and a pitching platform in a connection relationship; the adjusting seat is used for fixing and mounting a target or a plane plate; the translation track, the lifting rod, the rotating seat and the pitching platform are respectively used for adjusting the horizontal position, the vertical position, the rotating angle and the pitching angle of the target or the plane plate, and can respectively record the pose change of the rotating, the translation and the pitching angle. The method realizes the movement of the target or the plane plate in the calibration space by combining the camera external parameter calibration device with the target / plane plate, and calibrates the external parameters of two cameras. The method does not need to introduce large auxiliary equipment, is suitable for calibrating the external parameters of cameras with large spacing and almost deviated lenses, reduces the requirement on the calibration environment, is simple in design, easy to build, low in cost and high in calibration efficiency.

Description

Technical Field

[0001] The present invention relates to the field of camera calibration, and in particular to a device and method for calibrating extrinsic parameters of cameras with large spacing and no common field of view. Background Art

[0002] Currently, visual inspection has been widely used in the field of intelligent manufacturing. Due to the limited field of view of a single camera, when measuring large workpieces (such as high-speed trains, cars, ships, and airplanes), multiple cameras are needed to expand the field of view. When there are multiple cameras, the extrinsic parameters between the cameras need to be calibrated, and the calibration results of the camera extrinsic parameters directly affect the accuracy of the visual inspection results.

[0003] Existing camera extrinsic calibration methods typically find or set landmarks within the cameras' common field of view, using the same landmarks in the image as constraints to obtain the rotation and translation matrices between the two cameras. However, this method fails when the cameras do not have a common field of view. To address this issue, existing solutions require additional auxiliary equipment:

[0004] For example, patent document CN114373019A discloses a method for calibrating cameras without a common field of view using an optimization method. A third camera K with a common field of view with both the left and right cameras is added as an intermediary to obtain the three-dimensional coordinates of each marker point within the common field of view I and the common field of view II in the camera K coordinate system. When the camera lens to be calibrated is facing almost opposite directions, for example: in a wheelset detection system, Figure 1 As shown in the figure, camera 1 and camera 2 are installed on the inner side of the left and right tracks respectively, with the lenses facing the outside of the tracks. The angle of their optical axes is greater than 150°, and the distance between the two cameras is more than 50 cm. In this case, the placement of the third camera will be very far away, which will increase the difficulty of selecting the marking point and reduce the calibration accuracy.

[0005] For example, in his paper "Research and Application of Multi-Sensor Machine Vision Measurement Systems," Luo Ming introduced a dual-theodolite system to establish a spatial three-dimensional coordinate measurement system, directly measuring the three-dimensional coordinates of control points on the optical plane and achieving global calibration of a multi-camera vision measurement system. Patent document CN112308926A also introduced a photogrammetry system to assist in measuring the spatial coordinates of coded points, thereby obtaining extrinsic parameters between cameras with no common field of view. Such methods require the introduction of high-precision measurement equipment, which is expensive. Furthermore, when the cameras are installed outdoors, such as when cameras 1 and 2 of a wheelset detection sensor are installed on the inner sides of the left and right train tracks, the calibration site is limited and the environment is poor, making it unsuitable for the introduction of large-scale auxiliary calibration equipment. Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a camera extrinsic parameter calibration device and method for large spacing and no common field of view, which can realize the movement of a target or a plane panel in a calibration space, can image in different camera fields of view, can obtain the extrinsic parameter matrix between cameras without a common field of view without introducing large auxiliary equipment, and can adapt to the extrinsic parameter calibration between cameras with large spacing and almost opposite lenses, reduce the requirements for the calibration environment, and have the characteristics of simple design, easy to build, low cost and high calibration efficiency.

[0007] Therefore, the technical scheme of the present application is as follows:

[0008] A camera extrinsic parameter calibration device for large spacing and no common field of view, the camera has two parts, the positions are fixed, the spacing is at least 50 cm, the lenses are opposite, and the included angle between the optical axes is 120°-180°.

[0009] The camera extrinsic parameter calibration device comprises a translation track and an adjusting seat mounted on the translation track, the adjusting seat can slide on the translation track, and comprises a lifting rod, a rotating seat and a pitching platform in a connecting relationship; and the adjusting seat is used for fixedly mounting the target or the plane panel.

[0010] The translation track, the lifting rod, the rotating seat and the pitching platform are respectively used for adjusting the horizontal position, the vertical position, the rotation angle and the pitching angle of the target or the plane panel, and can respectively record the pose change amount of the rotation, the translation and the pitching angle.

[0011] The present application also discloses a method for calibrating cameras with large spacing and no common field of view by using the camera extrinsic parameter calibration device, the target is mounted on the adjusting seat, during calibration, the target pose is adjusted so that one camera can clearly collect the target image, the pose of the adjusting seat at this time is recorded as pose I, the rotation and translation relationship between the camera and the current target coordinate system is solved according to the coordinates of a plurality of mark points in the camera coordinate system and the target coordinate system in the collected target image.

[0012] The adjusting seat is moved, the target pose is adjusted so that the other camera can clearly image, the pose of the adjusting seat at this time is recorded as pose II, the rotation and translation relationship between the camera and the current target coordinate system is solved according to the coordinates of a plurality of mark points in the camera coordinate system and the target coordinate system in the target image collected by the camera.

[0013] The rotation and translation relationship between the two cameras is obtained based on the pose change amount recorded by the camera extrinsic parameter calibration device from pose I to pose II, and is recorded as an extrinsic parameter matrix.

[0014] Further, the mark points at least include 6 chessboard corner points, concentric circles or circles.

[0015] The origin of the target coordinate system is set at the geometric center of the target or at a corner point of the target.

[0016] The present invention also discloses another method for calibrating cameras with large spacing and no common field of view using a camera extrinsic parameter calibration device, wherein a multi-line laser is fixed on one side of each camera, and the rotational and translational relationship between the camera and the multi-line laser is pre-calibrated; the method is characterized in that:

[0017] The plane plate is installed on the adjustment seat; a global coordinate system is pre-established, and the plane equation of the plane plate in the initial position in the global coordinate system is recorded as the initial plane equation;

[0018] The calibration method steps are as follows:

[0019] S1. Adjust the flat panel to the field of view of a camera. Project multiple laser strips onto the surface of the flat panel using a multi-line laser. Adjust the position and orientation of the laser strips so that the camera can capture a clear image of the laser strips. Select multiple 3D points on different laser strips in the laser strip image to fit the spatial plane equation of the flat panel in the camera coordinate system, which is recorded as plane equation A1.

[0020] Record the rotation, translation, and pitch angle changes of the current position of the adjustment base relative to the initial position. Based on the position changes and the initial plane equation, solve the spatial plane equation of the plane plate at the current position in the global coordinate system, which is recorded as plane equation A2;

[0021] The posture of the plane plate is changed multiple times, and each time the posture is changed, a pair of plane equations A1 and A2 are obtained;

[0022] Solve multiple pairs of plane equations A1 and A2 to obtain the rotation and translation relationship between the camera coordinate system and the global coordinate system;

[0023] S2. Repeat step S1 for another camera to obtain the rotation and translation relationship between the coordinate system of the other camera and the global coordinate system;

[0024] S3. Using the global coordinate system as a medium, obtain the rotation and translation relationship between the two cameras, which is recorded as the extrinsic parameter matrix.

[0025] Furthermore, there are two ways to calculate the initial plane equation:

[0026] Method 1: Select at least three non-collinear feature points on the plane plate, including corner points, edge points, and holes; obtain the three-dimensional coordinates of the multiple feature points in the global coordinate system at the initial position according to the processing size of the plane plate, and use the obtained multiple three-dimensional coordinates to fit the spatial plane equation, which is recorded as the initial plane equation;

[0027] Method 2: At the initial position, according to the processing size of the plane plate, obtain the normal vector of the plane plate in the global coordinate system and the three-dimensional coordinates of a single feature point in the global coordinate system, and use the normal vector and the three-dimensional coordinates of the feature point to fit the spatial plane equation and record it as the initial plane equation.

[0028] Furthermore, the spatial plane equation of the plane plate at the current position in the global coordinate system is solved, which is recorded as plane equation A2; the method is as follows:

[0029] Let the coefficients of the initial plane equation be [AB CD];

[0030] Then the coefficients of the plane equation A2 are [A'B'C'D'] = [ABCD] · [RT1] -1 ;

[0031] Among them, RT1 represents the rotation and translation relationship obtained based on the rotation, translation and pitch angle changes of the current posture of the adjustment seat relative to the initial posture, which is composed of the rotation angle adjustment change α, the pitch angle adjustment change β, the horizontal translation change Δx and the vertical translation change Δh.

[0032] Preferably, the plane plate is a polygonal plane plate or a circular plane plate with holes on the surface;

[0033] It is preferably a regular polygonal plane plate, such as a square plane plate, a regular triangle plane plate, or a regular pentagon;

[0034] The surface of the flat plate is pure color without pattern, and the grayscale difference between its color and the color of the laser bar is greater than 100.

[0035] Further, the multi-line laser projects at least two laser strips;

[0036] The origin of the global coordinate system is set at a spatial point that does not change in position.

[0037] Furthermore, in step S1, the posture of the plane plate is changed at least three times, and a pair of plane equations A1 and A2 are obtained each time the posture is changed.

[0038] Furthermore, the translation track is fixedly installed in the calibration site; the lifting rod is vertically installed on the translation track and can slide left and right on the translation track;

[0039] The rotating seat is installed on the lifting rod, and the lifting rod is used to adjust the height of the rotating seat; the pitching platform is installed on the rotating seat, and the rotating seat is used to rotate the installation angle of the pitching platform; the pitching platform is used to install the target / plane board and adjust the pitch angle of the target / plane board;

[0040] Alternatively, the pitch platform is installed on a lifting rod, and the lifting rod is used to adjust the height of the pitch platform; the rotating seat is installed on the pitch platform, and the pitch platform is used to adjust the pitch angle of the rotating seat; the rotating seat is used to install the target / plane board and adjust the installation angle of the target / plane board.

[0041] The present invention has the following beneficial effects:

[0042] ① No need to introduce expensive large-scale auxiliary equipment, thus reducing equipment costs;

[0043] The camera extrinsic calibration device enables the target or plane plate to be movable in the calibration space and imaged in different camera fields of view. By knowing the transformation relationship under different postures, the extrinsic parameter matrix between cameras without a common field of view is obtained.

[0044] ② The requirements for the calibration environment are reduced and it can be carried out outdoors. The camera extrinsic parameter calibration device is simple in design, easy to build, and has high calibration efficiency.

[0045] ③ It is capable of extrinsic calibration between two cameras whose lenses are almost opposite to each other and have a large distance between them; for example Figure 1 The wheelset detection system shown has two sensors (containing cameras) installed on the left and right tracks respectively, forming multiple pairs of camera groups without a common field of view, and the lenses of the two cameras without a common field of view are almost opposite to each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of a wheelset detection system in the prior art;

[0047] Figure 2 Schematic diagram of the structure of the camera extrinsic calibration device in Example 1;

[0048] Figure 3 Schematic diagram of the process of calibrating two cameras in Example 1;

[0049] Figure 4 Schematic diagram of the process of calibrating four cameras in Example 1;

[0050] Figure 5 Schematic diagram of the process of calibrating two cameras in Example 2;

[0051] Figure 6 Schematic diagram of the process of calibrating four cameras in Example 2. DETAILED DESCRIPTION

[0052] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0053] Example 1

[0054] A device for calibrating external parameters of cameras with large spacing and no common field of view. The device has two fixed cameras with a spacing of at least 50 cm. The lenses face opposite directions and the angle between the optical axes is 120° to 180°.

[0055] like Figure 2 As shown, the camera extrinsic parameter calibration device includes a translation track 1 and an adjustment seat mounted on the translation track. The adjustment seat can slide on the translation track and includes a lifting rod 2, a rotating seat 3 and a pitching platform 4 in a connected relationship. In this embodiment, the adjustment seat is used to fix the target 5.

[0056] The translation track 1, lifting rod 2, rotating seat 3 and pitching platform 4 are used to adjust the horizontal position, vertical position, rotation angle and pitch angle of the target or plane plate respectively, and can record the position changes of rotation, translation and pitch angle respectively.

[0057] In specific implementation, the translation track can be made of high-precision linear transmission robot seventh axis ground rail, guide rail with encoder, electric cylinder, linear translation stage, etc.

[0058] The lifting rod can be made of a straight displacement linear slide, an electric vertical displacement table, an electric cylinder, etc.

[0059] The rotating seat can be equipped with high-precision vacuum rotating table, rotating pan / tilt table, electric angle table, etc.

[0060] The pitch stage can be equipped with a high-precision electric angular stage, a pitch and yaw stage, or an optical experiment arc tilt stage.

[0061] In this embodiment, a target is used to calibrate the external parameters between cameras without a common field of view, wherein the translation track is fixedly installed in the calibration site (around the two cameras); the lifting rod 2 is vertically installed on the translation track 1 and can slide left and right on the translation track 1;

[0062] The rotating seat is installed on the lifting rod 2, and the lifting rod 2 is used to adjust the height of the rotating seat 3; the pitching platform 4 is installed on the rotating seat, and the rotating seat is used to rotate the installation angle of the pitching platform; the pitching platform is used to install the target 5 and adjust the pitch angle of the target;

[0063] Alternatively, the pitching platform 4 is installed on the lifting rod 2, and the lifting rod 2 is used to adjust the height of the pitching platform 4; the rotating seat 3 is installed on the pitching platform, and the pitching platform is used to adjust the pitch angle of the rotating seat; the rotating seat is used to install the target 5 and adjust the installation angle of the target.

[0064] The specific calibration method is as follows:

[0065] A method for calibrating cameras with large spacing and no common field of view using a camera extrinsic calibration device, wherein a target is mounted on an adjustment base, and during calibration, the target position is adjusted, such as Figure 3, record the pose of the adjusting seat as pose I, and solve the rotation and translation relationship RT between the camera and the current target coordinate system according to the coordinates of the plurality of mark points in the target image collected by the camera in the camera coordinate system and the target coordinate system A ;

[0066] Move the adjusting seat to adjust the pose of the target, so that the other camera can clearly image, record the pose of the adjusting seat as pose II, and solve the rotation and translation relationship RT between the camera and the current target coordinate system according to the coordinates of the plurality of mark points in the target image collected by the camera in the camera coordinate system and the target coordinate system B ;

[0067] Based on the pose change amount from pose I to pose II recorded by the camera external parameter calibration device, the rotation and translation relationship between the two cameras is obtained, which is recorded as an external parameter matrix.

[0068] Specifically, based on the pose change amount from pose I to pose II recorded by the camera external parameter calibration device, the rotation and translation relationship RT between pose I and pose II is obtained;

[0069] The rotation and translation relationship RT is composed of a rotation angle change amount α, a pitch angle adjustment change amount β, a change amount Δx along the horizontal direction, and a change amount Δh along the vertical direction.

[0070] For example, the camera external parameter calibration device first rotates counterclockwise along the central axis of the pitch table by β, then rotates counterclockwise along the central axis of the rotating seat by α, then moves along the horizontal direction by Δx, and moves along the vertical direction by Δh, then

[0071]

[0072] The rotation and translation relationship between the two cameras is obtained through RT as a medium, which is recorded as an external parameter matrix.

[0073] That is, the external parameter matrix between camera one and camera two = RT A × RT × RT B -1 .

[0074] Among them, since there are 12 unknown parameters in RT A / RT B , at least 6 pairs of mark points are needed for constraint, therefore, the mark points in the target at least include 6, which are chessboard corner points, concentric circles or circles; usually, more than 10 mark points are set.

[0075] The origin of the target coordinate system is set at the geometric center of the target or at the corner point of the target.

[0076] As an extended application of this embodiment, the solution of this embodiment is applied to the simultaneous calibration of four cameras, such as Figure 4 The four cameras form multiple pairs of camera groups without a common field of view. Their layout is roughly rectangular. The translation track is installed on the symmetry axis of the rectangle. Each pair of camera groups without a common field of view is calibrated using the target.

[0077] The specific process is as follows:

[0078] The four cameras to be calibrated are fixed in position. The optical axes of cameras 1, 2, 3, and 4 are almost facing in opposite directions. The distance between any two cameras is more than 50 cm. There is no common field of view between the four cameras to be calibrated.

[0079] Use the camera external parameter calibration device to move the target to the target pose A. At this pose, camera 1 captures the target image and obtains the rotation and translation relationship T between camera 1 and the current target coordinate system. Ato1 ;

[0080] Use the camera external parameter calibration device to move the target to the target pose B. At this pose, camera 2 captures the target image and obtains the rotation and translation relationship T between camera 2 and the current target coordinate system. Bto2 ;

[0081] Use the camera external parameter calibration device to move the target to the target pose C. At this pose, camera three captures the target image and obtains the rotation and translation relationship T between camera three and the current target coordinate system. Cto3 ;

[0082] Use the camera external parameter calibration device to move the target to the target pose D. At this pose, camera 4 captures the target image and obtains the rotation and translation relationship T between camera 4 and the current target coordinate system. Dto4 ;

[0083] The data recorded by the camera external parameter calibration device (including the movement data of the translation track 1, the movement data of the lifting rod 2, the rotation angle of the rotating seat 3, and the rotation angle of the pitch platform 4) are used to calculate the posture transformation matrix T when moving from target posture A to target posture B, target posture A to target posture C, and target posture A to target posture D. AtoB 、T AtoC 、T AtoD ;

[0084] Then the coordinate system rotation and translation relationships (external parameters) between camera 1 and camera 2, camera 1 and camera 3, and camera 1 and camera 4 are:

[0085] T 21 =T 2toB T BtoA T Ato1 =T Bto2 -1 TAtoB -1 T Ato1

[0086] T 31 =T 3toC T CtoA T Ato1 =T Cto3 -1 T AtoC -1 T Ato1

[0087] T 41 =T 4toD T DtoA T Ato1 =T Dto4 -1 T AtoD -1 T Ato1

[0088] As a subsequent application, the information collected by cameras two, three, and four is converted to the coordinate system of camera one through the external parameters between the four cameras, thus realizing the coordinate system one of the collected information.

[0089] Example 2

[0090] In this embodiment, the structure of the camera extrinsic parameter calibration device is the same as that of Example 1, except that the adjustment base is used to fix the flat plate; that is, the translation track is fixedly installed in the calibration site (around the two cameras); the lifting rod is vertically installed on the translation track and can slide left and right on the translation track;

[0091] The swivel seat is installed on the lifting rod, and the lifting rod is used to adjust the height of the swivel seat; the pitching platform is installed on the swivel seat, and the swivel seat is used to rotate the installation angle of the pitching platform; the pitching platform is used to install the plane board and adjust the pitch angle of the plane board;

[0092] Alternatively, the pitching platform is installed on the lifting rod, and the lifting rod is used to adjust the height of the pitching platform; the rotating seat is installed on the pitching platform, and the pitching platform is used to adjust the pitch angle of the rotating seat; the rotating seat is used to install the plane board and adjust the installation angle of the plane board.

[0093] In this embodiment, a flat plate is used to calibrate the extrinsic parameters between cameras without a common field of view. A multi-line laser is attached to one side of each camera, and the rotational and translational relationships between the camera and the attached multi-line laser are pre-calibrated. This method is also applicable to calibrating the extrinsic parameters between multi-line structured light sensors, where the sensor housing includes a camera and a multi-line laser fixed in relative position.

[0094] The specific calibration method is as follows:

[0095] A method for calibrating cameras with large spacing and no common field of view using a camera extrinsic parameter calibration device comprises the following steps:

[0096] A plane plate is installed on the adjustment seat; a global coordinate system is established in advance, and the plane equation of the plane plate in the initial position in the global coordinate system is recorded as the initial plane equation;

[0097] The calibration steps are as follows:

[0098] S1. Adjust the plane board to the field of view of a camera, such as Figure 5 As shown, a multi-line laser projects multiple laser strips onto the surface of a plane plate, adjusts the position, and the camera captures a clear laser strip image. In the laser strip image, multiple three-dimensional points on different laser strips are selected to fit the spatial plane equation of the plane plate in the camera coordinate system, which is recorded as plane equation A1.

[0099] Record the rotation, translation, and pitch angle changes of the current position of the adjustment base relative to the initial position. Based on the position changes and the initial plane equation, solve the spatial plane equation of the plane plate at the current position in the global coordinate system, which is recorded as plane equation A2;

[0100] The posture of the plane plate is transformed multiple times, and each time the posture is transformed, a pair of plane equations A1 and A2 are obtained;

[0101] Solve multiple pairs of plane equations A1 and A2 to obtain the rotation and translation relationship between the camera coordinate system and the global coordinate system;

[0102] Since the rotational and translational relationship between the camera coordinate system and the global coordinate system involves six degrees of freedom and twelve unknown parameters, and a pair of plane equations provides four equations, at least three pairs of plane equations are required to complete the solution. To this end, in step S1, the plane plate's pose is transformed at least three times, and each pose transformation yields a pair of plane equations A1 and A2.

[0103] S2. Repeat step S1 for another camera to obtain the rotation and translation relationship between the coordinate system of the other camera and the global coordinate system;

[0104] S3. Using the global coordinate system as a medium, obtain the rotation and translation relationship between the two cameras, which is recorded as the extrinsic parameter matrix.

[0105] The plane plate is a polygonal plane plate or a circular plane plate with holes on the surface; preferably, it is designed to be a regular polygonal plane plate, such as a square plane plate, a regular triangle plane plate, or a regular pentagon;

[0106] The surface of the flat panel is pure color without pattern, and the grayscale difference between its color and the color of the laser bar is greater than 100.

[0107] Multi-line lasers project at least two laser strips;

[0108] The origin of the global coordinate system is set at a point in space that does not change position, such as the geometric center or edge of the translation track.

[0109] Specifically, there are two ways to calculate the initial plane equation:

[0110] Method 1: Select at least three non-collinear feature points on the plane plate, including corner points, edge points, and holes; obtain the three-dimensional coordinates of multiple feature points in the global coordinate system at the initial position according to the processing size of the plane plate, and use the obtained multiple three-dimensional coordinates to fit the spatial plane equation, which is recorded as the initial plane equation;

[0111] For example: for a square plane plate, select four corner points and obtain the coordinates of the four corner points in the global coordinate system; use the coordinates of the four non-collinear points to obtain the spatial plane equation.

[0112] Method 2: At the initial position, according to the processing size of the plane plate, obtain the normal vector of the plane plate in the global coordinate system and the three-dimensional coordinates of a single feature point in the global coordinate system, and use the normal vector and the three-dimensional coordinates of the feature point to fit the spatial plane equation and record it as the initial plane equation.

[0113] For example: the origin of the global coordinate system is set at the geometric center of the translation track and the initial position of the camera extrinsic calibration device. The plane plate is placed horizontally with its geometric center facing the origin of the global coordinate system. At this time, the normal vector of the plane plate in the global coordinate system is (0, 0, 1); then the angle or hole on the plane plate is selected as the feature point, the three-dimensional coordinates of the single feature point in the global coordinate system are calculated, and the spatial plane equation is obtained using the point method.

[0114] Furthermore, the spatial plane equation of the plane plate at the current position in the global coordinate system is solved, which is recorded as plane equation A2; the method is as follows:

[0115] Let the coefficients of the initial plane equation be [AB CD];

[0116] The initial plane equation is: AX+BY+CZ+D=0;

[0117] Then the coefficients of the plane equation A2 are [A'B'C'D'] = [ABCD] · [RT1] -1 ;

[0118] Among them, RT1 is the rotation and translation relationship obtained based on the rotation, translation, and pitch angle changes of the current posture recorded by the adjustment seat relative to the initial posture, which is composed of the rotation angle adjustment change α, the pitch angle adjustment change β, the horizontal translation change Δx, and the vertical translation change Δh.

[0119] For example, the camera extrinsic calibration device first rotates β counterclockwise along the center axis of the pitch stage, then rotates α counterclockwise along the center axis of the rotating seat, and then moves Δx in the horizontal direction and Δh in the vertical direction.

[0120]

[0121] As an extended application of this embodiment, the solution of this embodiment is applied to a wheelset detection system, such as Figure 1 As shown in the figure, the four detection sensors are distributed in a rectangular shape and are all distributed on the inner side of the train track. Each visual sensor is equipped with a camera and a multi-line laser with relatively fixed positions. The four cameras form multiple pairs of camera groups without a common field of view, and the rotation and translation relationships (external parameter matrix) between the camera coordinate systems in the four sensors are calibrated.

[0122] like Figure 6 As shown, the translation track is installed on the symmetry axis of the rectangle. During implementation, a clamping mechanism is set under the translation track. The clamping mechanism is used to clamp the left train track and the right train track respectively, so that the camera extrinsic parameter calibration device lies horizontally on the two train tracks, and each pair of cameras without a common field of view is calibrated separately using a flat plate.

[0123] The specific process is as follows:

[0124] The four cameras to be calibrated are fixed in position. The optical axes of cameras 1, 2, 3, and 4 are almost facing in opposite directions. The distance between any two cameras is more than 50 cm. There is no common field of view between the four cameras to be calibrated.

[0125] Calibrate camera 1:

[0126] The camera extrinsic calibration device is used to move the plane plate to the target pose A. Laser 1 projects multiple laser lines onto the plane plate, and camera 1 collects images. The pose is changed multiple times near the pose (at least three times) to form a target pose set A1. Each time the pose is changed, a pair of plane equations A1 and A2 are obtained. Multiple pairs of plane equations A1 and A2 are jointly calculated as follows:

[0127]

[0128] Obtain the rotation and translation relationship T between the camera coordinate system and the global coordinate system wto1 The left side of the equation represents the equation coefficients of multiple plane equations A1, and the right side of the equation represents the equation coefficients of multiple plane equations A2. The equation coefficients of plane equation A2 are calculated as follows:

[0129] Let the coefficients of the initial plane equation be [AB CD];

[0130] Then the coefficients of the plane equation A2 are [A'B'C'D'] = [ABCD] · [RT1] -1 ;

[0131] Among them, RT1 is the rotation and translation relationship obtained based on the rotation, translation, and pitch angle changes of the current posture recorded by the adjustment seat relative to the initial posture, which is composed of the rotation angle adjustment change α, the pitch angle adjustment change β, the horizontal translation change Δx, and the vertical translation change Δh.

[0132] In the above equations, the matrix T wto1 There are 6 degrees of freedom and 12 unknown parameters. A pair of plane equations can provide 4 equations. When more than 3 pairs of plane equations are obtained, the above overdetermined linear equations can be solved.

[0133] Calibrate camera 2:

[0134] The plane plate is moved to the target pose B using the camera extrinsic calibration device. Laser 2 projects multiple laser lines onto the plane plate, and camera 2 collects images. The pose is changed multiple times near this pose (at least three times) to form a target pose set B1. Each time the pose is changed, a pair of plane equations A1 and A2 are obtained. Multiple pairs of plane equations A1 and A2 are combined to obtain the rotation and translation relationship T between the camera 2 coordinate system and the global coordinate system. wto2 ;

[0135] Calibrate camera three:

[0136] The camera extrinsic calibration device is used to move the plane plate to the target pose C. The laser projects multiple laser lines on the plane plate in three directions, and the camera collects images. The pose is changed multiple times near the pose (at least three times) to form a target pose set C1. Each time the pose is changed, a pair of plane equations A1 and A2 are obtained. Multiple pairs of plane equations A1 and A2 are combined to obtain the rotation and translation relationship T between the camera three-coordinate system and the global coordinate system. wto3 ;

[0137] Calibrate camera four:

[0138] The camera extrinsic calibration device is used to move the plane plate to the target pose D. The laser four-axis plane plate projects multiple laser lines, and the camera four collects images. The pose is changed multiple times near the pose (at least three times) to form a target pose set D1. Each time the pose is changed, a pair of plane equations A1 and A2 are obtained. Multiple pairs of plane equations A1 and A2 are combined to obtain the rotation and translation relationship T between the camera four-axis coordinate system and the global coordinate system. wto4 ;

[0139] External parameters between cameras:

[0140] The coordinate system rotation and translation relationship (external parameter) between camera one and camera two, camera one and camera three, camera one and camera four are respectively T 21 , 31 , 41 :

[0141] T 21 = T 2tow T wto1 = T 2tow -1 T wto1

[0142] T 31 = T 3tow T wto1 = T 3tow -1 T wto1

[0143] T 41 = T 4tow T wto1 = T 4tow -1 T wto1

[0144] As a subsequent application of the external parameter matrix, through the external parameters between the four cameras, the information collected by cameras two, three and four is respectively converted to the coordinate system of camera one, so as to realize the coordinate system one of the collected information.

[0145] For the convenience of explanation and accurate definition of the appended claims, the terms "upper", "lower", "inner" and "outer" are used to describe the features of the exemplary embodiments with reference to the positions of the features shown in the drawings.

[0146] The foregoing description of specific exemplary embodiments of the application presented herein is not intended to be exhaustive or to be limited to the precise forms of the application described above. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the application be limited not with the specific exemplary embodiments presented above, but rather by the claims appended hereto.

Claims

1. A method for calibrating cameras with a large spacing and no common field of view using a camera extrinsic calibration device, wherein the cameras are fixed in position, separated by at least 50 cm, with their lenses facing opposite directions and their optical axes at an angle of 120° to 180°. Its characteristics are: The camera extrinsic parameter calibration device includes a translation track and an adjustment seat installed on the translation track, the adjustment seat can slide on the translation track, and includes a lifting rod, a rotating seat and a pitching platform with a connection relationship; the adjustment seat is used to fix the target; The translation track, lifting rod, rotating seat and pitching platform are used to adjust the horizontal position, vertical position, rotation angle and pitch angle of the target respectively, and can record the position changes of rotation, translation and pitch angle respectively; During calibration, the target pose is adjusted so that a camera can clearly capture the target image. The pose of the adjustment base at this time is recorded as pose I. Based on the coordinates of multiple landmarks in the captured target image in the camera coordinate system and the target coordinate system, the rotation and translation relationship between the camera and the current target coordinate system is solved. Move the adjustment base and adjust the target pose so that the other camera can clearly image. The pose of the adjustment base at this time is recorded as pose II. According to the coordinates of multiple landmarks in the target image collected by the camera in the camera coordinate system and the target coordinate system, solve the rotation and translation relationship between the camera and the current target coordinate system; Based on the pose change from pose I to pose II recorded by the camera extrinsic calibration device, the rotation-translation relationship RT between pose I and pose II is obtained. The rotation-translation relationship RT is composed of the rotation angle change α, the pitch angle adjustment change β, the horizontal translation change Δx, and the vertical translation change Δh. Then the rotation and translation relationship between the two cameras is obtained and recorded as the external parameter matrix.

2. The method according to claim 1, wherein: The marking points include at least 6, which are checkerboard corner points, concentric circles or circles; The origin of the target coordinate system is set at the geometric center of the target or at a corner point of the target.

3. A method for calibrating cameras with a large spacing and no common field of view using a camera extrinsic calibration device, wherein the cameras are fixed in position, separated by at least 50 cm, with their lenses facing opposite directions and their optical axes at an angle of 120° to 180°. Its characteristics are: The camera extrinsic parameter calibration device includes a translation track and an adjustment seat installed on the translation track. The adjustment seat can slide on the translation track and includes a lifting rod, a rotating seat and a pitching platform with a connection relationship; the adjustment seat is used to fix the installation plane plate; The translation track, lifting rod, rotating seat and pitching platform are used to adjust the horizontal position, vertical position, rotation angle and pitch angle of the plane board respectively, and can record the position changes of rotation, translation and pitch angle respectively; A multi-line laser is fixed on one side of each camera, and the rotation and translation relationship between the camera and the multi-line laser is pre-calibrated; a global coordinate system is pre-established, and the plane equation of the plane plate in the initial position in the global coordinate system is recorded as the initial plane equation; The calibration method steps are as follows: S1. Adjust the flat panel to the field of view of a camera. Project multiple laser strips onto the surface of the flat panel using a multi-line laser. Adjust the position and orientation of the laser strips so that the camera can capture a clear image of the laser strips. Select multiple 3D points on different laser strips in the laser strip image to fit the spatial plane equation of the flat panel in the camera coordinate system, which is recorded as plane equation A1. Record the rotation, translation, and pitch angle changes of the current position of the adjustment base relative to the initial position. Based on the position changes and the initial plane equation, solve the spatial plane equation of the plane plate at the current position in the global coordinate system, which is recorded as plane equation A2; The posture of the plane plate is changed multiple times, and each time the posture is changed, a pair of plane equations A1 and A2 are obtained; Solve multiple pairs of plane equations A1 and A2 to obtain the rotation and translation relationship between the camera coordinate system and the global coordinate system; S2. Repeat step S1 for another camera to obtain the rotation and translation relationship between the coordinate system of the other camera and the global coordinate system; S3. Using the global coordinate system as a medium, obtain the rotation and translation relationship between the two cameras, which is recorded as the extrinsic parameter matrix.

4. The method according to claim 1 or 3, wherein: The translation track is fixedly installed in the calibration site; the lifting rod is vertically installed on the translation track and can slide left and right on the translation track; The rotating seat is installed on the lifting rod, and the lifting rod is used to adjust the height of the rotating seat; the pitching platform is installed on the rotating seat, and the rotating seat is used to rotate the installation angle of the pitching platform; the pitching platform is used to install the target / plane board and adjust the pitch angle of the target / plane board; Alternatively, the pitching platform is mounted on a lifting rod, and the lifting rod is used to adjust the height of the pitching platform; The rotating seat is installed on a pitch platform, and the pitch platform is used to adjust the pitch angle of the rotating seat; the rotating seat is used to install the target / plane board and adjust the installation angle of the target / plane board.

5. The method according to claim 3, wherein: There are two ways to calculate the initial plane equation: Method 1: Select at least three non-collinear feature points on the plane plate, including corner points, edge points, and holes; obtain the three-dimensional coordinates of the multiple feature points in the global coordinate system at the initial position according to the processing size of the plane plate, and use the obtained multiple three-dimensional coordinates to fit the spatial plane equation, which is recorded as the initial plane equation; Method 2: At the initial position, according to the processing size of the plane plate, obtain the normal vector of the plane plate in the global coordinate system and the three-dimensional coordinates of a single feature point in the global coordinate system, and use the normal vector and the three-dimensional coordinates of the feature point to fit the spatial plane equation and record it as the initial plane equation.

6. The method according to claim 3, wherein: Solve the spatial plane equation of the plane plate in the global coordinate system at the current pose, denoted as plane equation A2; the method is as follows: The coefficients of the initial plane equation are recorded as [ABCD]; Then the coefficients of the plane equation A2 [A' B' C' D'] = [ABCD] · [RT1] -1 ; Among them, RT1 represents the rotation and translation relationship obtained based on the rotation, translation and pitch angle changes of the current posture of the adjustment seat relative to the initial posture, which is composed of the rotation angle adjustment change α, the pitch angle adjustment change β, the horizontal translation change Δx and the vertical translation change Δh.

7. The method according to claim 3, wherein: The plane plate is a polygonal plane plate or a circular plane plate with holes on the surface.

8. The method according to claim 3, wherein: The plane plate is a regular polygonal plane plate, including: a square plane plate, a regular triangle plane plate and a regular pentagonal plane plate; The surface of the flat plate is pure color without pattern, and the grayscale difference between its color and the color of the laser bar is greater than 100.

9. The method according to claim 3, wherein: The multi-line laser projects at least two laser strips; The origin of the global coordinate system is set at a spatial point that does not change in position.

10. The method according to claim 3, wherein: In step S1, the posture of the plane plate is changed at least three times, and a pair of plane equations A1 and A2 are obtained each time the posture is changed.

Images