A semi-automatic calibration method for binocular stereo camera

Abstract

The application discloses a kind of binocular stereo camera's semi-automatic calibration method, through camera shooting inlayed in visual positioning mark of calibration template pattern, obtain the three-dimensional space position and pose of camera in real world, then generate virtual projection mark in calibration template pattern, accurately guide calibration personnel to adjust camera position and angle, make virtual projection mark real-time tracking guide mark movement, record reasonable, left and right camera image pair containing calibration template pattern, finally call calibration function to extract the feature point of calibration template pattern and determine the internal and external parameters of camera.The calibration method has the advantages of high precision, high efficiency, easy operation, simple structure of calibration system, low cost, strong universality, suitable for large-scale production of binocular stereo camera calibration work, with high engineering application value.

Description

TECHNICAL FIELD

[0001] The application relates to a calibration method of a binocular stereo camera, in particular to a semi-automatic calibration method of a binocular stereo camera. BACKGROUND

[0002] A binocular stereo camera needs to calibrate its internal and external parameters, including lens focal length, principal point position, distortion coefficient, rotation matrix and translation vector between cameras, before use. In use, image distortion correction and stereo correction are performed according to these parameters to achieve the purpose of accelerating disparity map calculation and three-dimensional reconstruction.

[0003] At present, most binocular stereo camera calibrations use special calibration objects. The main calibration steps include: collecting calibration object images; determining the positions of feature points of the calibration object in the real world coordinate system and the camera image coordinate system; and determining the internal and external parameters of the camera according to the camera projection model and the positions of the feature points. The latter two steps have been very mature in research and application, such as the famous “Zhang Zhengyou calibration method”, which has been widely used as a packaged function or toolbox. Therefore, in actual engineering applications, the main work of binocular stereo camera calibration is to collect the images of the logical calibration object, and then only the ready-made calibration function needs to be called.

[0004] According to the intervention degree of the calibration personnel in the process of collecting the calibration object images, the binocular stereo camera calibration method can be divided into manual calibration and automatic calibration.

[0005] Manual calibration requires the calibration personnel to frequently move the camera or the calibration object, so that the camera can shoot images containing the calibration object from multiple different positions and angles. This method requires high experience of the calibration personnel, and it is easy to cause poor calibration accuracy or even calibration failure due to unreasonable shooting images, and the calibration process is very complicated and time-consuming, so it is only suitable for limited number of binocular camera calibration.

[0006] Automatic calibration basically does not require human intervention, has high automation degree and fast calibration speed, and is suitable for calibration of large-scale mass-produced binocular stereo cameras. However, this method requires one or more sets of special motion control devices to drive the camera or the calibration object to different positions for shooting. Therefore, a special motion control device needs to be designed in advance according to the nominal parameters of the camera and the size of the calibration object, which leads to the problems of complex calibration system, great design difficulty, high cost and poor universality, so there are very few practical application cases at present. SUMMARY

[0007] The application aims to provide a semi-automatic calibration method of a binocular stereo camera, which solves the problems of poor accuracy and low efficiency of the existing manual calibration method, and the problems of complex calibration system, great design difficulty, high cost and poor universality of the automatic calibration method.

[0008] The steps of a semi-automatic calibration method of a binocular stereo camera are:

[0009] The first step is to construct a semi-automatic calibration system of a binocular stereo camera

[0010] The semi-automatic calibration system of the binocular stereo camera comprises a calibration template determination module, an image acquisition module, a camera positioning module, a virtual projection mark determination module, a guide mark determination module, and a calibration parameter determination module.

[0011] The calibration template determination module, the image acquisition module, the camera positioning module, the virtual projection mark determination module, the guide mark determination module, and the calibration parameter determination module all run in a computer.

[0012] The function of the calibration template determination module is to create a calibration template pattern and display it on the computer display screen.

[0013] The function of the image acquisition module is to acquire the binocular image of the computer display screen photographed by the binocular stereo camera in real time, and output it to the camera positioning module, the calibration parameter determination module, and the computer display screen.

[0014] The function of the camera positioning module is to determine the three-dimensional space position and attitude of the binocular stereo camera in the display screen coordinate system.

[0015] The function of the virtual projection mark determination module is to determine the center and diameter of the virtual projection mark circular spot in the checkerboard picture coordinate system, and superimpose it on the checkerboard pattern and display it on the computer display screen.

[0016] The function of the guide mark determination module is to determine the center position and moving speed of the guide mark in the moving state in the checkerboard picture coordinate system, and superimpose it on the checkerboard pattern and display it on the computer display screen.

[0017] The function of the calibration parameter determination module is to determine the camera internal and external parameters.

[0018] The second step is for the calibration template determination module to determine and display the calibration template pattern

[0019] The calibration template determination module creates the calibration template pattern and displays it on the computer display screen. The calibration template pattern adopts a checkerboard pattern, which is embedded with camera positioning marks and initial guide marks.

[0020] The calibration template pattern is in the display screen coordinate system O W X W Y W Z Wwidth and height in the display screen coordinate system are respectively denoted as W and H, and the chessboard pattern is displayed in the display screen by a chessboard pattern picture with resolution of M x N. In order to ensure the calibration accuracy, when the chessboard pattern is simultaneously complete in the left camera image and the right camera image, the horizontal axis of the chessboard pattern is substantially parallel to the horizontal frame of the display screen, and the binocular camera is closest to the display screen, the following conditions are met: the proportion of the chessboard width in the image width is not less than μ, and the proportion of the chessboard height in the image height is not less than v, at this time, the chessboard width and height in the display screen coordinate system are respectively denoted as minimum width W min and minimum height H min , W min and H min are expressed by formula (1):

[0021]

[0022] In formula (1), B is the nominal length of the binocular camera baseline, θ and are the horizontal and vertical field angles of the camera respectively, and α is the included angle between the left end point of the horizontal axis of the chessboard and the optical axis of the left camera. The determination of θ,

[0023]

[0024] In formula (2), X res and Y res are the horizontal and vertical resolutions of the camera image sensor respectively, f is the nominal focal length of the camera lens in millimeters, f x and f y are the nominal focal length of the camera lens in pixels, d x and d y are the horizontal and vertical physical sizes of a single pixel of the camera image sensor respectively, and the unit is the same as f.

[0025] W and H meet the condition expressed by formula (3):

[0026]

[0027] In formula (3), W monitor and H monitor are the width and height of the computer display screen respectively, X max x Y max is the highest resolution of the display screen, and d m is the physical size of the display screen pixel.

[0028] After W and H are determined, the resolution M x N of the chessboard pattern picture is expressed by formula (4):

[0029]

[0030] Each grid of the checkerboard is square, and there are UxV grids in total, where U and V are both positive integers and satisfy The side length of the grid is L grid It is expressed by formula (5):

[0031]

[0032] The camera positioning mark is three black solid circles, respectively embedded in three white checkerboard grids that are not collinear in fixed positions. In the checkerboard picture coordinate system O P X P Y P The centers of the camera positioning marks are denoted as {C i , i = 1, 2, 3}, which coincide with the centers of the corresponding grids, and the diameter of the solid circle is D m = 0.8L grid , and the distance between the centers satisfies: In the display screen coordinate system O W X W Y W Z W The centers of the camera positioning marks are denoted as:

[0033] The initial guide mark is a fixed-size circular ring that is stationary at the center of the calibration template pattern, and in the checkerboard picture coordinate system O P X P Y P The inner diameter of the guide mark circular ring is R The outer diameter is R When the virtual projection mark first coincides with the initial guide mark, the initial guide mark becomes a moving guide mark and moves along a fixed route until the stop moving condition is met.

[0034] The display screen coordinate system O W X W Y W Z W Definition: The origin O W is located at the center of the checkerboard pattern in the computer display screen, the X-axis is parallel to the horizontal axis of the checkerboard and points to the right of the display screen, the Y-axis is parallel to the vertical axis of the checkerboard and points to the lower part of the display screen, and the Z-axis is determined by the right-hand system, perpendicular to the display screen plane and pointing to the back of the display screen.

[0035] The checkerboard picture coordinate system O P X P Y P Definition: The origin O P is located at the leftmost upper corner pixel of the checkerboard picture with resolution MxN, the X-axis is parallel to the horizontal direction of the picture and points to the right, and the Y-axis is parallel to the vertical direction of the picture and points to the lower part. The coordinate system is in units of pixels.

[0036] Third step Image acquisition module acquires and displays computer display screen pattern

[0037] The image acquisition module acquires the binocular image of the computer display screen taken by the binocular stereo camera in real time, and outputs to the camera positioning module, the calibration parameter determination module and the computer display screen.

[0038] Fourth step Camera positioning module determines the spatial position and attitude of the camera

[0039] The camera positioning module determines the three-dimensional spatial position and attitude of the binocular stereo camera in the display screen coordinate system.

[0040] After the camera positioning module receives the real-time binocular image, it identifies all the camera positioning marks of the left and right image pairs by using the Hough circle transformation method, and obtains the positions of the circle centers in the left and right camera image coordinate systems And The positions of these circle centers in the left camera coordinate system O L X L Y L Z L It can be expressed by formula (6):

[0041]

[0042] In formula (6), d i represents the disparity of each circle center in the left and right images, which can be expressed by formula (7):

[0043]

[0044] Determine the three-dimensional spatial position and attitude of the camera in the display screen coordinate system O W X W Y W Z W , that is, the rotation matrix R and the translation vector t between the left camera coordinate system O L X L Y L Z L and the display screen coordinate system O W X W Y W Z W , which can be expressed by formula (8):

[0045] P i W = RP i L +t, i=1,2,3 (8)

[0046] The left camera coordinate system O L X​L Y L Z L Definition: Origin O L Located at the optical center of the left camera, the X-axis is parallel to the horizontal axis of the image coordinate system and points to the right, the Y-axis is parallel to the vertical axis of the image coordinate system and points downward, and the Z-axis is the optical axis of the left camera and points forward.

[0047] Camera image coordinate system O C X C Y C Definition: Origin O C Located at a resolution of X res ×Y res The coordinate system is defined by the top-left pixel of the image captured by the camera. The X-axis is parallel to the horizontal axis of the image and points to the right, while the Y-axis is parallel to the vertical axis and points downwards. This coordinate system is in pixels.

[0048] The fifth step, the virtual projection mark determination module, determines and displays the virtual projection mark.

[0049] The virtual projection marker determination module determines the circular light spot of the virtual projection marker in the coordinate system O of the checkerboard image. P X P Y P The center Q in P =(u q ,v q ) T and diameter D v It is overlaid onto the checkerboard pattern and displayed on the computer screen.

[0050] The virtual projection mark simulates a circular light spot formed on the display screen by projecting a beam of light from the center point C of the camera baseline. Its position and size depend on the orientation and position of the stereo camera. During calibration, the calibration personnel hold the stereo camera, align it with the center of the checkerboard calibration template, and photograph it, ensuring that the virtual projection mark on the display screen roughly coincides with the initial guide mark. After the guide mark begins to move, the calibration personnel change the camera's position and orientation, align it with the guide mark on the display screen, and photograph it again, so that the virtual projection mark tracks the guide mark and roughly coincides with it, achieving the purpose of photographing the checkerboard from different positions and angles.

[0051] The virtual projection marker determination module first determines the virtual projection marker based on {P} i L Determine the X coordinate system of the display screen. W O W Y W The plane equation in the left camera coordinate system is π:A p x+B p y+C p z+D p =0, plane equation coefficients A p Bp and C p That is, the plane normal vector The three components, Expressed using formula (9), P1 L Substituting π into the plane equation will determine the coefficient D. p .

[0052]

[0053] Point C emits a virtual ray parallel to the Z-axis of the left camera coordinate system in front of the camera, intersecting with the X-axis of the display screen coordinate system. W O W Y W The planes intersect at point Q, which is the center of the virtual projected circular light spot. The position of Q in the left camera coordinate system is... Expressed using formula (10):

[0054]

[0055] Q in the display coordinate system O W X W Y W Z W The position in the middle Expressed using formula (11):

[0056] Q W =RQ L +t (11)

[0057] Q is in the coordinate system O of the chessboard image. P X P Y P Position Q in P =(u q ,v q ) T Expressed using formula (12):

[0058]

[0059] Circular spot diameter D v Length of line segment CQ Inversely proportional, expressed by formula (13):

[0060]

[0061] In formula (13), It is O P X P Y P The inner diameter of the guide mark ring in the coordinate system.

[0062] The sixth step, the guide marker determination module, determines and displays the guide markers indicating the movement status.

[0063] The guidance mark determination module determines the movement state of the guidance marks in the checkerboard image coordinate system O. P X P Y P The center position of the circle The movement speed V is superimposed onto the checkerboard pattern and displayed on the computer screen.

[0064] The moving guide marker is a fixed-size circular ring that moves continuously along a custom, fixed, and closed trajectory. It guides the calibration personnel to change the camera's position and orientation, causing the virtual projection marker to track and essentially overlap with the guide marker. The guide marker stops moving once the checkerboard image acquisition termination condition is met.

[0065] The calibration personnel hold a camera and aim it at the guide mark on the display screen, adjusting the camera's position and angle so that the virtual projection mark appears on the screen, tracking the guide mark and roughly aligning it with it. When the virtual projection mark is smaller than the guide mark, the calibration personnel move the camera closer to the display screen; otherwise, they move it away from the display screen.

[0066] The trajectory of the moving guide markers consists entirely of straight line segments, including horizontal, vertical, and diagonal lines, uniformly covering the entire checkerboard pattern. The trajectory is divided into K sub-trajectories of different line segment types, denoted as {L}. i Let i = 1, 2, ..., K, where 10 ≤ K ≤ 50. Sub-lines are divided into two types: unrecorded and recorded. All sub-lines are initially set to the unrecorded type. When the guide marker moves on any unrecorded sub-line, if the virtual projection marker and the guide marker first substantially overlap and the checkerboard pattern simultaneously appears completely in the left and right camera image pairs, the left and right camera image pairs at this time are recorded, and then the sub-line is set to the recorded type. When the number of recorded sub-lines is not less than 0.8K, the checkerboard image acquisition termination condition is met, and the guide marker stops moving.

[0067] The moving speed of the guide marker V = {V i Let V = {i = 1, 2, ..., K}, where V i The guide mark is on sub-line L i The speed of movement on V i Expressed using formula (14):

[0068]

[0069] Center of the guide mark circle The initial position of the guide mark is in the center of the checkerboard pattern and is stationary. The guide mark starts to move when the virtual projection mark and the guide mark first substantially coincide. When the guide mark moves, Depending on the sub-line L i where the guide mark is currently located i and the time T i when it has entered L i . The current sub-line L i is uniquely determined by its start point and end point , and the vector is expressed by equation (15):

[0070]

[0071] is expressed by equation (16):

[0072]

[0073] The condition for the virtual projection mark and the guide mark to be substantially coincident is expressed by equation (17):

[0074]

[0075] Step 7: The calibration parameter determination module determines the calibration parameters

[0076] The calibration parameter determination module determines the camera internal and external parameters. When the guide mark stops moving, the checkerboard image acquisition is completed. The calibration parameter determination module first uses the OpenCV checkerboard corner extraction function to find all the checkerboard corners in the recorded left and right camera image pairs, obtains their positions in the left and right camera image pairs, and then combines their fixed positions in the real world to determine the camera internal and external parameters using the OpenCV stereo calibration function. The camera internal and external parameter calibration results are not only saved to the computer storage medium, but also transmitted to the camera for storage and use. Finally, the camera is powered off and disconnected from the computer.

[0077] At this point, the semi-automatic calibration of the binocular stereo camera is completed.

[0078] The advantages of the present application compared to previous binocular stereo camera calibration methods are mainly: high calibration accuracy, high efficiency, strong universality, low calibration system cost, suitable for large-scale mass production of binocular stereo cameras for fast and accurate calibration, and high engineering application value. DETAILED DESCRIPTION

[0079] The steps of a semi-automatic calibration method for a binocular stereo camera are:

[0080] First step: constructing a semi-automatic calibration system of binocular stereo camera

[0081] The semi-automatic calibration system of binocular stereo camera comprises a calibration template determination module, an image acquisition module, a camera positioning module, a virtual projection mark determination module, a guide mark determination module and a calibration parameter determination module.

[0082] The calibration template determination module, the image acquisition module, the camera positioning module, the virtual projection mark determination module, the guide mark determination module and the calibration parameter determination module all run in the computer.

[0083] The function of the calibration template determination module is to create a calibration template pattern and display it on the computer display screen.

[0084] The function of the image acquisition module is to acquire the binocular image of the computer display screen shot by the binocular stereo camera in real time, and output it to the camera positioning module, the calibration parameter determination module and the computer display screen.

[0085] The function of the camera positioning module is to determine the three-dimensional space position and attitude of the binocular stereo camera in the display screen coordinate system.

[0086] The function of the virtual projection mark determination module is to determine the center and diameter of the virtual projection mark circular spot in the checkerboard picture coordinate system, and superimpose it to the checkerboard pattern and display it on the computer display screen.

[0087] The function of the guide mark determination module is to determine the center position and moving speed of the guide mark in the moving state in the checkerboard picture coordinate system, and superimpose it to the checkerboard pattern and display it on the computer display screen.

[0088] The function of the calibration parameter determination module is to determine the camera internal and external parameters.

[0089] Second step: the calibration template determination module determines and displays the calibration template pattern

[0090] The calibration template determination module creates a calibration template pattern and displays it on the computer display screen. The calibration template pattern adopts a checkerboard pattern, which is embedded with camera positioning marks and initial guide marks.

[0091] The calibration template pattern is in the display screen coordinate system O W X W Y W Z Wwidth and height in the display screen coordinate system are respectively denoted as W and H, and the chessboard pattern is displayed in the display screen by a picture with resolution of M x N. In order to ensure the calibration accuracy, when the chessboard pattern appears completely in the left camera image and the right camera image, the horizontal axis of the chessboard pattern is substantially parallel to the horizontal frame of the display screen, and the distance between the binocular camera and the display screen is the shortest, the following conditions are met: the proportion of the chessboard width in the image width is not less than μ, and the proportion of the chessboard height in the image height is not less than v. At this time, the chessboard width and height in the display screen coordinate system are respectively denoted as minimum width W min and minimum height H min , W min and H min are expressed by formula (1):

[0092]

[0093] In formula (1), B is the nominal length of the baseline of the binocular camera, θ and are the horizontal and vertical field angles of the camera respectively, and α is the angle between the left end point of the horizontal axis of the chessboard and the optical axis of the left camera. The determination of θ, and α is expressed by formula (2):

[0094]

[0095] In formula (2), X res and Y res are the horizontal and vertical resolutions of the camera image sensor respectively, f is the nominal focal length of the camera lens in millimeters, f x and f y are the nominal focal length of the camera lens in pixels, d x and d y are the horizontal and vertical physical sizes of a single pixel of the camera image sensor respectively, and the unit is the same as f.

[0096] W and H meet the condition expressed by formula (3):

[0097]

[0098] In formula (3), W monitor and H monitor are the width and height of the computer display screen respectively, X max x Y max is the highest resolution of the display screen, and d m is the physical size of the display screen pixel.

[0099] After W and H are determined, the resolution M x N of the chessboard picture is expressed by formula (4):

[0100]

[0101] Each grid of the checkerboard is square, and there are UxV grids in total, where U and V are both positive integers and satisfy The side length of the grid is L grid It is expressed by formula (5):

[0102]

[0103] The camera positioning mark is three black solid circles, which are embedded in three white checkerboard grids that are not collinear in fixed positions. In the checkerboard picture coordinate system O P X P Y P The centers of the camera positioning marks are denoted as {C i , i = 1, 2, 3}, which coincide with the centers of the corresponding grids, and the diameter of the solid circle is D m = 0.8L grid , and the distance between the centers satisfies: In the display screen coordinate system O W X W Y W Z W The centers of the camera positioning marks are denoted as:

[0104] The initial guide mark is a fixed-size circular ring that is stationary at the center of the calibration template pattern. In the checkerboard picture coordinate system O P X P Y P The inner diameter of the guide mark circular ring is R The outer diameter is R When the virtual projection mark first coincides with the initial guide mark, the initial guide mark becomes a moving guide mark and moves along a fixed route until the stop moving condition is met.

[0105] The display screen coordinate system O W X W Y W Z W Definition: The origin O W is located at the center of the checkerboard pattern in the computer display screen, the X-axis is parallel to the horizontal axis of the checkerboard and points to the right of the display screen, the Y-axis is parallel to the vertical axis of the checkerboard and points to the lower part of the display screen, and the Z-axis is determined by the right-hand system, perpendicular to the display screen plane and pointing to the back of the display screen.

[0106] The checkerboard picture coordinate system O P X P Y P Definition: The origin O P is located at the leftmost upper corner pixel of the checkerboard picture with resolution MxN, the X-axis is parallel to the horizontal direction of the picture and points to the right, and the Y-axis is parallel to the vertical direction of the picture and points to the lower part. The coordinate system is in units of pixels.

[0107] Third step Image acquisition module acquires and displays computer display screen pattern

[0108] The image acquisition module acquires the binocular image of the computer display screen taken by the binocular stereo camera in real time, and outputs to the camera positioning module, the calibration parameter determination module and the computer display screen.

[0109] Fourth step Camera positioning module determines the spatial position and attitude of the camera

[0110] The camera positioning module determines the three-dimensional spatial position and attitude of the binocular stereo camera in the display screen coordinate system.

[0111] After the camera positioning module receives the real-time binocular image, it identifies all the camera positioning marks of the left and right image pairs by using the Hough circle transformation method, and obtains the positions of the circle centers in the left and right camera image coordinate systems And The positions of these circle centers in the left camera coordinate system O L X L Y L Z L are represented by formula (6):

[0112]

[0113] In formula (6), d i represents the disparity of each circle center in the left and right images, which is represented by formula (7):

[0114]

[0115] Determine the three-dimensional spatial position and attitude of the camera in the display screen coordinate system O W X W Y W Z W , that is, the rotation matrix R and the translation vector t between the left camera coordinate system O L X L Y L Z L and the display screen coordinate system O W X W Y W Z W , which is represented by formula (8):

[0116] P i W =RP i L +t, i=1,2,3 (8)

[0117] The left camera coordinate system O L X​L Y L Z L Definition: Origin O L Located at the optical center of the left camera, the X-axis is parallel to the horizontal axis of the image coordinate system and points to the right, the Y-axis is parallel to the vertical axis of the image coordinate system and points downward, and the Z-axis is the optical axis of the left camera and points forward.

[0118] Camera image coordinate system O C X C Y C Definition: Origin O C Located at a resolution of X res ×Y res The coordinate system is defined by the top-left pixel of the image captured by the camera. The X-axis is parallel to the horizontal axis of the image and points to the right, while the Y-axis is parallel to the vertical axis and points downwards. This coordinate system is in pixels.

[0119] The fifth step, the virtual projection mark determination module, determines and displays the virtual projection mark.

[0120] The virtual projection marker determination module determines the circular light spot of the virtual projection marker in the coordinate system O of the checkerboard image. P X P Y P The center Q in P =(u q ,v q ) T and diameter D v It is overlaid onto the checkerboard pattern and displayed on the computer screen.

[0121] The virtual projection mark simulates a circular light spot formed on the display screen by projecting a beam of light from the center point C of the camera baseline. Its position and size depend on the orientation and position of the stereo camera. During calibration, the calibration personnel hold the stereo camera, align it with the center of the checkerboard calibration template, and photograph it, ensuring that the virtual projection mark on the display screen roughly coincides with the initial guide mark. After the guide mark begins to move, the calibration personnel change the camera's position and orientation, align it with the guide mark on the display screen, and photograph it again, so that the virtual projection mark tracks the guide mark and roughly coincides with it, achieving the purpose of photographing the checkerboard from different positions and angles.

[0122] The virtual projection marker determination module first determines the virtual projection marker based on {P} i L Determine the X coordinate system of the display screen. W O W Y W The plane equation in the left camera coordinate system is π:A p x+B p y+C p z+D p =0, plane equation coefficients A p Bp and C p That is, the plane normal vector The three components, Expressed using formula (9), P1 L Substituting π into the plane equation will determine the coefficient D. p .

[0123]

[0124] Point C emits a virtual ray parallel to the Z-axis of the left camera coordinate system in front of the camera, intersecting with the X-axis of the display screen coordinate system. W O W Y W The planes intersect at point Q, which is the center of the virtual projected circular light spot. The position of Q in the left camera coordinate system is... Expressed using formula (10):

[0125]

[0126] Q in the display coordinate system O W X W Y W Z W The position in the middle Expressed using formula (11):

[0127] Q W =RQ L +t (11)

[0128] Q is in the coordinate system O of the chessboard image. P X P Y P Position Q in P =(u q ,v q ) T Expressed using formula (12):

[0129]

[0130] Circular spot diameter D v Length of line segment CQ Inversely proportional, expressed by formula (13):

[0131]

[0132] In formula (13), It is O P X P Y P The inner diameter of the guide mark ring in the coordinate system.

[0133] The sixth step, the guide marker determination module, determines and displays the guide markers indicating the movement status.

[0134] The guidance mark determination module determines the movement state of the guidance marks in the checkerboard image coordinate system O. P X P Y P The center position of the circle The movement speed V is superimposed onto the checkerboard pattern and displayed on the computer screen.

[0135] The moving guide marker is a fixed-size circular ring that moves continuously along a custom, fixed, and closed trajectory. It guides the calibration personnel to change the camera's position and orientation, causing the virtual projection marker to track and essentially overlap with the guide marker. The guide marker stops moving once the checkerboard image acquisition termination condition is met.

[0136] The calibration personnel hold a camera and aim it at the guide mark on the display screen, adjusting the camera's position and angle so that the virtual projection mark appears on the screen, tracking the guide mark and roughly aligning it with it. When the virtual projection mark is smaller than the guide mark, the calibration personnel move the camera closer to the display screen; otherwise, they move it away from the display screen.

[0137] The trajectory of the moving guide markers consists entirely of straight line segments, including horizontal, vertical, and diagonal lines, uniformly covering the entire checkerboard pattern. The trajectory is divided into K sub-trajectories of different line segment types, denoted as {L}. i Let i = 1, 2, ..., K, where 10 ≤ K ≤ 50. Sub-lines are divided into two types: unrecorded and recorded. All sub-lines are initially set to the unrecorded type. When the guide marker moves on any unrecorded sub-line, if the virtual projection marker and the guide marker first substantially overlap and the checkerboard pattern simultaneously appears completely in the left and right camera image pairs, the left and right camera image pairs at this time are recorded, and then the sub-line is set to the recorded type. When the number of recorded sub-lines is not less than 0.8K, the checkerboard image acquisition termination condition is met, and the guide marker stops moving.

[0138] The moving speed of the guide marker V = {V i Let V = {i = 1, 2, ..., K}, where V i The guide mark is on sub-line L i The speed of movement on V i Expressed using formula (14):

[0139]

[0140] Center of the guide mark circle the initial position of the guide mark is in the center of the checkerboard pattern and is stationary, and the guide mark starts to move when the virtual projection mark and the guide mark first substantially coincide in the position. When the guide mark moves, Depending on the current sub-line L i of the guide mark i and the time T i that has entered L i . The current sub-line L i is uniquely determined by its starting point and end point , and the vector is expressed by formula (15):

[0141]

[0142] is expressed by formula (16):

[0143]

[0144] The condition for the virtual projection mark and the guide mark to be substantially coincident is expressed by formula (17):

[0145]

[0146] Step 7: The calibration parameter determination module determines the calibration parameters

[0147] The calibration parameter determination module determines the camera internal and external parameters. When the guide mark ends moving, the checkerboard image acquisition ends. The calibration parameter determination module first uses the OpenCV checkerboard corner extraction function to find all the checkerboard corners in all the recorded left and right camera image pairs, obtains their positions in the left and right camera image pairs, and then combines the known fixed positions in the real world to determine the camera internal and external parameters using the OpenCV stereo calibration function. The camera internal and external parameter calibration results are not only saved to the computer storage medium, but also transmitted to the camera for storage and use. Finally, the camera is powered off and shut down, and the connection between the camera and the computer is disconnected.

[0148] At this point, the semi-automatic calibration of the binocular stereo camera is completed.

Claims

1. A semi-automatic calibration method of a binocular stereo camera, characterized in that The specific steps are as follows: First step: constructing a semi-automatic calibration system of a binocular stereo camera The semi-automatic calibration system of the binocular stereo camera comprises a calibration template determination module, an image acquisition module, a camera positioning module, a virtual projection marker determination module, a guide marker determination module and a calibration parameter determination module. The calibration template determination module, the image acquisition module, the camera positioning module, the virtual projection marker determination module, the guide marker determination module and the calibration parameter determination module all run in a computer. Second step: the calibration template determination module determines and displays a calibration template pattern The calibration template determination module creates a calibration template pattern and displays it on a computer display screen; the calibration template pattern adopts a checkerboard pattern, and the checkerboard pattern is embedded with a camera positioning marker and an initial guide marker. Width and height of the calibration template pattern in the display screen coordinate system O W X W Y W Z W are respectively W and H, which are displayed on the display screen by a checkerboard picture with resolution MxN; to ensure the calibration accuracy, when the checkerboard pattern appears completely in the left camera image, the right camera image, the horizontal axis of the checkerboard pattern is basically parallel to the horizontal frame of the display screen, and the binocular camera is closest to the display screen, it satisfies that the proportion of the checkerboard width in the image width is not less than μ, and the proportion of the checkerboard height in the image height is not less than v, at this time, the checkerboard width and height in the display screen coordinate system are respectively the minimum width W min and the minimum height H min , W min and H min are expressed by formula (1): In equation (1), B is the nominal length of the baseline of the binocular camera, θ and are the horizontal and vertical field of view angles of the cameras, respectively, and α is the angle between the line connecting the left end point of the horizontal axis of the chessboard and the optical center of the left camera and the optical axis of the left camera. The determination of θ, and α is expressed by equation (2). In formula (2), X res and Y res These are the horizontal and vertical resolutions of the camera's image sensor, respectively; f is the nominal focal length of the camera lens, measured in millimeters. x and f y The nominal focal length of a camera lens is measured in pixels (d). x and d y These are the horizontal and vertical physical dimensions of a single pixel on the camera's image sensor, respectively, and are in the same unit as f. W and H satisfy the condition represented by formula (3): In formula (3), W monitor and H monitor are the width and height of the computer display screen, respectively, X max × Y max is the highest resolution of the display screen, and d m is the physical size of the display screen pixels. After W and H are determined, the resolution M×N of the checkerboard picture is represented by formula (4): Each of the grids of the checkerboard is a square, and there are UxV grids in total, U and V are both positive integers and satisfy The side length of the grid is L grid This is expressed by formula (5): Camera positioning marks are three black solid circles embedded in three white checkerboard grids respectively, which are not collinear in fixed positions; in the checkerboard picture coordinate system O P X P Y P , the centers of the camera positioning marks are denoted as {C i ,i=1,2,3}, which coincide with the centers of the corresponding grids, and the diameter of the solid circle is D m =0.8L grid , the distance between each center satisfies: In the display screen coordinate system O W X W Y W Z W , the centers of the camera positioning marks are denoted as: {P i W =(X i W ,Y i W ,Z i W ) T ,i=1,2,3}. The initial guide mark is a fixed-size circular ring that is stationary at the center of the calibration template pattern, in the checkerboard picture coordinate system O P X P Y P In this case, the inner diameter of the guide mark circular ring The outer diameter When the virtual projection mark first coincides with the initial guide mark, the initial guide mark becomes a moving guide mark and moves along a fixed route until the stop moving condition is met. Display screen coordinate system O W X W Y W Z W Definition: origin O W Located in the center of the checkerboard pattern in the computer display, the X axis is parallel to the horizontal axis of the checkerboard pointing to the right of the display, the Y axis is parallel to the vertical axis of the checkerboard pointing to the lower part of the display, the Z axis is determined by the right hand system, perpendicular to the display plane and pointing to the back of the display Chessboard picture coordinate system O P X P Y P Definition: origin O P Located at the top-left corner pixel of the chessboard picture with resolution MxN, the X-axis is parallel to the horizontal direction of the picture and points to the right, and the Y-axis is parallel to the vertical direction of the picture and points downward; this coordinate system is in units of pixels; Third step: the image acquisition module acquires and displays a pattern on the computer display screen The image acquisition module acquires binocular images of the computer display screen photographed by the binocular stereo camera in real time, and outputs the binocular images to the camera positioning module, the calibration parameter determination module and the computer display screen. Fourth step: the camera positioning module determines the spatial position and posture of the camera The camera positioning module determines the three-dimensional spatial position and posture of the binocular stereo camera in the display screen coordinate system. After receiving real-time stereo images, the camera localization module uses the Hough circle transform to identify all camera localization markers in the left and right image pairs, obtaining the position of their center in the coordinate systems of the left and right camera images. and These centers are in the left camera coordinate system O. L X L Y L Z L The position in the middle Expressed using formula (6): In Equation (6), d i The disparity of each circle center in the left and right images is represented by Equation (7). determine the three-dimensional spatial position and pose of the camera in the display screen coordinate system O W X W Y W Z W , i.e. the rotation matrix R and the translation vector t between the left camera coordinate system O L X L Y L Z L and the display screen coordinate system O W X W Y W Z W are expressed by equation (8): P i W = RP i L + t,i = 1,2,3 (8) Left camera coordinate system O L X L Y L Z L Definition: origin O L Located at the left camera optical center, the X-axis is parallel to the horizontal axis of the image coordinate system and points to the right, the Y-axis is parallel to the vertical axis of the image coordinate system and points downward, and the Z-axis is the optical axis of the left camera and points forward. Camera image coordinate system O C X C Y C Definition: origin O C Located at the top-left pixel of the camera's captured image with resolution X res ×Y res , the X-axis is parallel to the horizontal direction of the picture and points to the right, and the Y-axis is parallel to the vertical direction of the picture and points downward; this coordinate system is in units of pixels; Fifth step: the virtual projection marker determination module determines and displays a virtual projection marker The virtual projection marker determination module determines the circular light spot of the virtual projection marker in the coordinate system O of the checkerboard image. P X P Y P The center Q in P =(u q ,v q ) T and diameter D v This is overlaid onto the checkerboard pattern and displayed on the computer screen; The virtual projection marker is a circular light spot formed by simulating a light beam projected from a center point C of a camera baseline to the display screen, and the position and size of the virtual projection marker depend on the orientation and position of the binocular stereo camera; when the calibration is started, the calibration personnel hold the binocular camera to aim at and photograph the center of the checkerboard calibration template, so that the virtual projection marker in the display screen is substantially coincident with the initial guide marker; after the guide marker starts to move, the calibration personnel change the position and posture of the camera to aim at and photograph the guide marker of the display screen, so that the virtual projection marker tracks and is substantially coincident with the guide marker, and the purpose of photographing the checkerboard at different positions and angles is achieved. The virtual projection mark determination module first determines the X i L} of the display screen coordinate system according to {P W O W Y W The plane equation in the left camera coordinate system is π: Ax p x+B p y+C p z+D p = 0, and the plane equation coefficients A p , B p , and C p are three components of the plane normal vector , which is expressed by formula (9). Then, by substituting P1 L into the plane equation π, the coefficient D p can be determined. Point C sends a virtual ray parallel to the Z axis of the left camera coordinate system to the front of the camera, which intersects the X W O W Y W The plane intersects at point Q, which is the center of the virtual projection mark circular light spot; the position of Q in the left camera coordinate system is It is expressed by formula (10): Q in the display screen coordinate system O W X W Y W Z W in the display screen coordinate system O is expressed by formula (11) Q W = RQ L + t (11) Q is the position of the chessboard picture coordinate system O P X P Y P in the position Q P = (u q , v q ) T is expressed by formula (12): Circular spot diameter D v is inversely proportional to the length of the line segment CQ is inversely proportional to the length of the line segment CQ In formula (13), is O P X P Y P Inner diameter of the guide mark annulus in the coordinate system; Sixth step: the guide marker determination module determines and displays a moving guide marker The guide mark determination module determines the center position of the guide mark of the moving state in the checkerboard picture coordinate system O P X P Y P in the checkerboard picture coordinate system O and the moving speed V, which are superimposed to the checkerboard pattern and displayed on the computer display screen; The moving guide marker is a fixed-size ring, which is continuously moved on a self-defined, fixed and closed track line, and guides the calibration personnel to change the position and posture of the camera, so that the virtual projection marker tracks and is substantially coincident with the guide marker; after the checkerboard image acquisition end condition is reached, the guide marker stops moving. The calibration personnel hold the camera to aim at the guide marker of the display screen, adjust the position and angle of the camera, so that the virtual projection marker appears on the display screen, tracks and is substantially coincident with the guide marker; when the virtual projection marker is smaller than the guide marker, the calibration personnel make the camera close to the display screen, otherwise, the calibration personnel make the camera away from the display screen. The trajectory line of the guide mark in the moving state is composed of straight line segments, including horizontal line segments, vertical line segments and inclined lines, and the ranges uniformly cover the entire checkerboard pattern; the line is divided into K segment line type sub-lines, denoted as {L i , i = 1, 2,..., K}, wherein 10≤K≤50; the sub-lines are divided into two types of unrecorded and recorded; the initial type of all the sub-lines is set to the unrecorded type; when the guide mark moves in any one unrecorded type sub-line, when the virtual projection mark and the guide mark first substantially coincide and the checkerboard pattern simultaneously appears in the left and right camera image pairs, the left and right camera image pairs at this time are recorded and saved, and then the sub-line is set to the recorded type; when the number of the recorded type sub-lines is not less than 0.8K, the checkerboard image acquisition end condition is reached, and the guide mark stops moving; The moving speed V of the guide mark = {V i , i = 1, 2,..., K}, wherein V i is the moving speed of the guide mark on the sub-line L i , V i is expressed by formula (14): Center of the guide mark circle The initial position is in the exact center of the checkerboard pattern and remains stationary. When the virtual projection mark and the guide mark first substantially overlap at this position, the guide mark begins to move; as the guide mark moves... Depends on the current sub-line L where the guidance marker is located i The current movement speed V of the guide marker i and has entered L i Time T i Current sub-line L i From its starting point and the end point Uniquely determined, vector Expressed using formula (15): is expressed by formula (16) The condition that the virtual projection marker and the guide marker are substantially coincident is represented by formula (17): Seventh step: the calibration parameter determination module determines calibration parameters The calibration parameter determination module determines the camera internal and external parameters; when the guide mark ends moving, the checkerboard image acquisition ends; the calibration parameter determination module firstly uses the OpenCV checkerboard corner extraction function to find all the checkerboard corners in the recorded left and right camera image pairs, obtains their positions in the left and right camera image pairs, combines the known fixed positions in the real world, and then uses the OpenCV stereo calibration function to determine the camera internal and external parameters. The camera internal and external parameter calibration results are not only saved to the computer storage medium, but also transmitted to the camera for saving and use; finally, the camera is powered off and shut down, and the connection between the camera and the computer is disconnected. Thus, the semi-automatic calibration of the binocular stereo camera is completed.

2. The semi-automatic calibration method of binocular stereo camera according to claim 1, characterized in that The function of the calibration template determination module is to create a calibration template pattern and display it on the computer display screen.

3. The semi-automatic calibration method of binocular stereo camera according to claim 1, characterized in that The function of the image acquisition module is to acquire the binocular image of the computer display screen photographed by the binocular stereo camera in real time, and output it to the camera positioning module, the calibration parameter determination module and the computer display screen.

4. The semi-automatic calibration method of binocular stereo camera according to claim 1, characterized in that The function of the camera positioning module is to determine the three-dimensional space position and attitude of the binocular stereo camera in the display screen coordinate system.

5. The semi-automatic calibration method of binocular stereo camera according to claim 1, characterized in that The function of the virtual projection mark determination module is to determine the center and diameter of the virtual projection mark circular light spot in the checkerboard picture coordinate system, superimpose it to the checkerboard pattern and display it on the computer display screen.

6. The semi-automatic calibration method of binocular stereo camera according to claim 1, characterized in that The function of the guide mark determination module is to determine the center position and moving speed of the guide mark in the moving state in the checkerboard picture coordinate system, superimpose it to the checkerboard pattern and display it on the computer display screen.

7. The semi-automatic calibration method of binocular stereo camera according to claim 1, wherein The function of the calibration parameter determination module is to determine the camera internal and external parameters.

Images