Binocular camera automatic calibration method capable of being used for line production

Abstract

The invention discloses a binocular camera automatic calibration method for line production, which comprises the following steps: fixing a checkerboard calibration plate, slowly moving a binocular camera along a sliding guide rail, and calibrating the camera; a database is initialized, all inner corner points are found out, coordinates of the inner corner points serve as a set of data samples to be stored in the database, and the number of the data samples is N1; dynamically updating the database, and searching the optimal data combination at the present stage; when the number of the database samples exceeds N2, the minimum calibration error is lower than a threshold value error3, and the coordinate difference values of the positions of the two cameras in the binocular module in the y-axis direction and the z-axis direction are both smaller than a threshold value T, it is regarded that calibration succeeds. According to the invention, image data with different distances and different angles from the checkerboard calibration plate can be collected, dynamic calibration is carried out, a database can be dynamically updated, data samples influencing calibration errors can be deleted, an optimal data combination at the present stage can be searched, and more accurate internal and external parameters of the binocular camera can be obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to an automatic calibration method for a binocular camera that can be used for assembly line operations. Background Art

[0002] Before the smart lawn mower can realize autonomous obstacle avoidance and edge-following functions, it needs to obtain accurate three-dimensional information of the current environment. The binocular stereo vision algorithm is one of the implementation solutions for obtaining three-dimensional information of images, and the three-dimensional information of the scene restored by the binocular stereo vision algorithm has a good effect. The first step is to obtain the accurate internal and external parameter matrix of the binocular camera, which needs to be obtained through a calibration algorithm.

[0003] The binocular camera calibration process usually includes: collection of checkerboard image data, inner corner detection, camera calibration, etc. The calibration effect is very easily affected by the image samples in the database. It is often necessary to repeat the calibration process many times to obtain a relatively accurate camera internal and external parameter matrix. This static calibration method is time-consuming, labor-intensive and extremely inefficient. It is obviously not suitable for enterprises to calibrate large quantities of binocular cameras in daily production processes, and the obtained camera parameters are not very accurate. Summary of the invention

[0004] In order to solve the above problems, the present invention provides an automatic calibration method for a binocular camera that can be used in assembly line operations, which can ensure the efficiency, stability and accuracy of the calibration process.

[0005] To this end, the technical solution of the present invention is: a binocular camera automatic calibration method that can be used for assembly line operations, comprising the following steps:

[0006] S1. Fix the checkerboard calibration plate on the workbench and install the binocular camera on the sliding rail;

[0007] S2, let the binocular camera move slowly along the sliding rail to perform camera calibration;

[0008] S3, database initialization, through corner point detection, find all the inner corner points in the checkerboard calibration plate image taken by the left and right cameras, and store their coordinates as a group of data samples in the database, repeat until the number of data samples in the database reaches N1 groups;

[0009] S4. The database is continuously and dynamically updated, adding new data samples that can reduce the standard error, and removing the data samples that most affect the calibration effect in the current database;

[0010] S5. If the number of database samples exceeds N2 groups, the minimum calibration error is lower than the threshold error3, and the coordinate differences of the two camera positions in the binocular module in the y-axis and z-axis directions are less than the threshold T, the calibration is considered successful; otherwise, if any condition is not met, jump to step S4 and continue the calibration work.

[0011] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S4, when the number of samples in the database reaches N1 groups, if the calculated calibration error decreases after the new data samples are added, then the group of data samples is retained and the value of the minimum calibration error is updated; otherwise, the group of data samples is eliminated.

[0012] On the basis of the above scheme and as a preferred scheme of the above scheme: the calculation method of the calibration error is as follows:

[0013] i. Obtain the internal and external parameters of the camera based on the calibration, and calculate the coordinates of the projection point of the target point on the image plane;

[0014] ii. Obtain the coordinates of the projection point actually observed on the image plane of the target point;

[0015] iii. Calculate the Euclidean distance between the actual observed projection point coordinates and the projection point coordinates calculated according to the calibration parameters;

[0016] iv. Normalization.

[0017] On the basis of the above scheme and as a preferred scheme of the above scheme: when a new data sample is added and the calibration error is lower than the threshold error2, the data sample is retained regardless of whether the calibration error is reduced.

[0018] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S4, at intervals of time t2, a cross-validation method is used to find out the data sample that most affects the calibration effect in the current database, and the data sample is eliminated.

[0019] Based on the above scheme and as a preferred scheme of the above scheme: the steps of data sample elimination are as follows:

[0020] ① All the data in the database participate in the calculation of camera calibration to obtain the calibration error e;

[0021] ② Traverse the n sets of data in the database. Except for the current data, all the remaining data participate in the calculation of camera calibration to obtain the calibration error e0, ..., e(n-1);

[0022] ③ Calculate the difference between e0, ..., e(n-1) and e respectively;

[0023] ④ Eliminate the set of data corresponding to the smallest difference from the database.

[0024] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S3, if the newly added data sample will cause the calibration error of the database to exceed a certain threshold error1, then the group of data is eliminated, otherwise it is retained.

[0025] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S3, if the initialization is not completed within a certain time threshold t1 seconds, the database is cleared and reinitialized.

[0026] On the basis of the above scheme and as a preferred scheme of the above scheme: in the step S2, the distance between the binocular camera and the chessboard calibration plate is between d1 and d2, and the binocular camera collects image data at different distances and angles from the chessboard calibration plate.

[0027] On the basis of the above scheme and as a preferred scheme of the above scheme: the initial position of the binocular camera is at the center of the sliding guide rail, and after the calibration starts:

[0028] S2.1, move along the X-axis to one end, rotate one circle with the center as the center, and return to the center;

[0029] S2.2, move along the X-axis to the other end, rotate one circle with the center as the center, and return to the center;

[0030] S2.3, move along the Y axis to one end, rotate around the center of the circle, and return to the center;

[0031] S2.4, move along the Y axis to the other end, rotate one circle with the center as the center, and return to the center;

[0032] S2.5. Repeat the above trajectory cycle until the calibration is successful.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] By setting the binocular camera on the sliding rail, image data at different distances and angles from the checkerboard calibration plate can be collected to implement dynamic calibration. During the calibration process, the database can be dynamically updated, data samples that will affect the calibration error can be deleted, and the optimal data combination at this stage can be found. This method has low cost and high efficiency, and can obtain more accurate internal and external parameters of the binocular camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a dynamic calibration flow chart of the present invention;

[0036] Figure 2 Schematic diagram of the movement of the binocular camera of the present invention.

[0037] Marked in the figure are: rotating seat 1, first linear motion module 2, second linear motion module 3, binocular camera 4. DETAILED DESCRIPTION

[0038] In the description of the present invention, it should be noted that directional words, such as the terms "center", "lateral (X)", "longitudinal (Y)", "vertical (Z)", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc., indicating directions and positional relationships are based on the directions or positional relationships shown in the accompanying drawings, which are only for the convenience of narrating the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and cannot be understood as limiting the specific scope of protection of the present invention.

[0039] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features. Therefore, the definition of "first" and "second" features can explicitly or implicitly include one or more of the features. In the description of the present invention, "several" and "a number" mean two or more, unless otherwise clearly and specifically defined.

[0040] See the attached drawings. The binocular camera automatic calibration method that can be used for assembly line operation described in this embodiment is characterized by:

[0041] S1. Fix the checkerboard calibration plate on the workbench, send the binocular camera to the designated position through the conveyor belt, connect the binocular camera to the industrial computer through the serial port, install the binocular camera on the sliding guide rail, and the binocular camera 4 can move in two directions, parallel to the checkerboard calibration plate and perpendicular to the checkerboard calibration plate.

[0042] For example: the sliding guide rail can be a rotating cross structure, and a first linear motion module 2 in the Y-axis direction and a second linear motion module 3 in the X-axis direction are installed above a rotating seat 1, and the binocular camera 4 is slidably installed on the second linear motion module 3. The rotating seat 1 can drive the first linear motion module 2 and the second linear motion module 3 to rotate, and the second linear motion module 3 can move along the first linear motion module 2 in the Y-axis direction, and the binocular camera 4 can move on the second linear motion module 3 in the X-axis direction.

[0043] S2. Move the binocular camera slowly along the slide rail to calibrate the camera. Run the calibration program stored on the industrial computer to move the binocular camera slowly along the slide rail until the camera is calibrated successfully. The binocular camera can move in a direction perpendicular to the checkerboard calibration board, and the distance from the checkerboard calibration board is d1 to d2 centimeters, so that the binocular camera can collect image data at different distances and angles from the checkerboard calibration board.

[0044] The initial position of the binocular camera is at the center of the slide rail. After calibration begins:

[0045] S2.1, the binocular camera moves the second linear movement module 3 along the X-axis to the far left, the rotating seat drives the binocular camera to rotate one circle, and then the binocular camera moves the second linear movement module 3 along the X-axis back to the center;

[0046] S2.2, the binocular camera moves along the second straight line moving module 3 to the far right, the rotating seat drives the binocular camera to rotate one circle, and then the binocular camera moves along the second straight line moving module 3 back to the center;

[0047] S2.3, the binocular camera is located in the middle of the second linear moving module 3, the second linear moving module 3 drives the binocular camera to move to the rear along the Y-axis of the first linear moving module 2, the rotating seat drives the binocular camera to rotate one circle, and then the second linear moving module 3 and the binocular camera return to the center along the first linear moving module 2;

[0048] S2.4, the second linear moving module 3 drives the binocular camera to move to the front along the Y axis of the first linear moving module 2, the rotating seat drives the binocular camera to rotate one circle, and then the second linear moving module 3 and the binocular camera return to the center along the first linear moving module 2;

[0049] S2.5. Repeat the above trajectory cycle until the calibration is successful.

[0050] S3, database initialization, through corner point detection, find all the inner corner points in the chessboard calibration plate image taken by the left and right cameras respectively, and store their coordinates as a group of data samples in the database, and repeat until the number of data samples in the database reaches N1 groups.

[0051] The corner detection here includes two processes: ① Using template matching technology to locate corners by comparing the similarity between the local area in the image and the predefined checkerboard template; ② defining a search window around the detected corner points, and calculating the gradient information to help determine the optimal position of the corner points, thereby obtaining the corner point coordinates with sub-pixel positioning accuracy.

[0052] If the newly added data sample causes the calibration error of the database to exceed a certain threshold error1, then the group of data will be removed, otherwise it will be retained. Generally, the value of error1 will be set slightly larger to ensure that the database can be initialized quickly and successfully.

[0053] If the initialization is not completed within a certain time threshold t1 seconds, the database will be cleared and reinitialized.

[0054] S4. Dynamic update of database to find the optimal data combination at this stage:

[0055] S4.1. When the number of samples in the database reaches N1 groups, if the calculated calibration error decreases after adding new data samples, then the group of data samples is retained and the value of the minimum calibration error is updated. Otherwise, the group of data samples is eliminated.

[0056] The normalized measure of the reprojection error is calculated as follows: ① Calculate the projection point coordinates of the target point on the image plane based on the camera internal and external parameters (camera imaging model) obtained through calibration; ② The projection point coordinates of the target point actually observed on the image plane; ③ Calculate the Euclidean distance between the actually observed projection point coordinates and the projection point coordinates calculated based on the calibration parameters; ④ Normalize.

[0057] S4.2, when the calibration error is lower than a very small threshold error2 after adding a new data sample, the data sample is retained regardless of whether the calibration error is reduced;

[0058] S4.3. Every once in a while t2, remove a group of data samples in the current database that have the greatest impact on the calibration effect; and use a cross-validation method to find the data samples to be deleted.

[0059] The specific steps of the cross-validation method are:

[0060] ① All the data in the database participate in the calculation of camera calibration to obtain the calibration error e;

[0061] ② Traverse the database (there are n sets of data). Except for the current data, all the remaining data participate in the calculation of camera calibration to obtain the calibration error e0,...,e(n-1);

[0062] ③ Calculate the difference between e0,...,e(n-1) and e respectively;

[0063] ④ Eliminate the group of data with the smallest difference from the database. For example, if the difference between e0 and e is the smallest, then eliminate the 0th group of data in the database.

[0064] S4.4, repeat the above steps until the calibration is successful;

[0065] S5. During the calibration process, if the following conditions are met at the same time, the camera internal and external parameters obtained by calibration are considered to meet the requirements and the calibration is considered successful:

[0066] ①The number of database samples exceeds N2 group;

[0067] ②The minimum calibration error is lower than the threshold error3; the value of threshold error3 is determined according to project requirements and is between threshold error1 and threshold error2. Generally, it is set to be less than 0.15 to better ensure the accuracy of the calibration results.

[0068] ③ The coordinate difference between the two camera positions in the binocular module in the y-axis and z-axis directions is less than the threshold T. For an ideal binocular module, the coordinate difference between the two camera positions in the y-axis and z-axis directions should be equal to 0. If the difference is too large, there are only two reasons: ① The production process level of the binocular module is not high (this reason can generally be directly excluded); ② The calibration error is large. Both of these reasons are unacceptable and such binocular modules should be avoided.

[0069] In this embodiment, the binocular camera is arranged on a sliding guide rail, which can move parallel or vertically relative to the checkerboard calibration plate, so as to collect image data at different distances and angles from the checkerboard calibration plate, and implement dynamic calibration. During the calibration process, the database can be dynamically updated, and data samples that will affect the calibration error can be deleted to find the optimal data combination at this stage. The cost is low and the efficiency is high, and more accurate internal and external parameters of the binocular camera can be obtained.

[0070] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A binocular camera automatic calibration method that can be used for assembly line operations, characterized in that: The following steps are involved: S1. Fix the checkerboard calibration plate on the workbench and install the binocular camera on the sliding rail; S2, let the binocular camera move slowly along the sliding rail to perform camera calibration; S3, database initialization, through corner point detection, find all the inner corner points in the checkerboard calibration plate image taken by the left and right cameras, and store their coordinates as a group of data samples in the database, repeat until the number of data samples in the database reaches N1 groups; S4. The database is continuously and dynamically updated, adding new data samples that can reduce the standard error, and removing the data samples that most affect the calibration effect in the current database; S5. If the number of database samples exceeds N2 groups, the minimum calibration error is lower than the threshold error3, and the coordinate differences of the two camera positions in the binocular module in the y-axis and z-axis directions are less than the threshold T, the calibration is considered successful; otherwise, if any condition is not met, jump to step S4 and continue the calibration work.

2. The method for automatic calibration of a binocular camera for assembly line operation as claimed in claim 1, characterized in that: In step S4, when the number of samples in the database reaches N1 groups, if the calculated calibration error decreases after adding new data samples, the group of data samples is retained and the value of the minimum calibration error is updated; otherwise, the group of data samples is removed.

3. The method for automatic calibration of a binocular camera for assembly line operation as claimed in claim 2, characterized in that: The calibration error is calculated as follows: i. Obtain the internal and external parameters of the camera based on the calibration, and calculate the coordinates of the projection point of the target point on the image plane; ii. Obtain the coordinates of the projection point actually observed on the image plane of the target point; iii. Calculate the Euclidean distance between the actual observed projection point coordinates and the projection point coordinates calculated according to the calibration parameters; iv. Normalization.

4. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 2, characterized in that: When a new data sample is added and the calibration error is lower than the threshold error2, the data sample is retained regardless of whether the calibration error is reduced.

5. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 1, characterized in that: In the step S4, at intervals of t2, a cross-validation method is used to find out the data sample in the current database that has the greatest impact on the calibration effect, and the data sample is removed.

6. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 5, characterized in that: The steps for removing data samples are as follows: ① All the data in the database participate in the calculation of camera calibration to obtain the calibration error e; ② Traverse the n sets of data in the database. Except for the current data, all the remaining data participate in the calculation of camera calibration to obtain the calibration error e0, ..., e(n-1); ③ Calculate the difference between e0, ..., e(n-1) and e respectively; ④ Eliminate the set of data corresponding to the smallest difference from the database.

7. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 1, characterized in that: In step S3, if the newly added data sample will cause the calibration error of the database to exceed a certain threshold error1, then the group of data is removed, otherwise it is retained.

8. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 1, characterized in that: In the step S3, if the initialization is not completed within a certain time threshold t1 seconds, the database is cleared and reinitialized.

9. The method for automatic calibration of a binocular camera applicable to assembly line operation as claimed in claim 1, characterized in that: In step S2, the distance between the binocular camera and the checkerboard calibration plate is between d1 and d2, and the binocular camera collects image data at different distances and angles from the checkerboard calibration plate.

10. The automatic calibration method for binocular cameras applicable to assembly line operations as claimed in claim 9, characterized in that: The initial position of the binocular camera is at the center of the slide rail. After calibration begins: S2.1, move along the X-axis to one end, rotate one circle with the center as the center, and return to the center; S2.2, move along the X-axis to the other end, rotate one circle with the center as the center, and return to the center; S2.3, move along the Y axis to one end, rotate around the center of the circle, and return to the center; S2.4, move along the Y axis to the other end, rotate one circle with the center as the center, and return to the center; S2.

5. Repeat the above trajectory cycle until the calibration is successful.

Images