Binocular line scanning camera calibration method and system

Abstract

The invention relates to the technical field of camera calibration, and discloses a binocular line-scan camera calibration method and system, the method calculates a position transformation matrix through a binocular line-scan camera calibration algorithm, can solve the problem of incomplete point cloud contour caused by shielding, further uses the position transformation matrix to fuse a first point cloud contour and a second point cloud contour, and improves the calibration accuracy of the binocular line-scan camera. And point cloud contour fusion at different angles is realized by adopting a direct calibration mode, so that the real-time performance of shielding scene measurement can be met, and the high efficiency of measurement calibration is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a binocular line scan camera calibration method and system. Background Art

[0002] Currently, in the gap and face difference detection technology based on 3D line scan cameras, most of them use a single line scan camera to obtain the point cloud contour. When encountering occlusion, it is necessary to adjust the camera or object angle to obtain the point cloud contour at this angle, and finally fuse the point clouds at multiple angles to obtain a complete point cloud for measurement. It can be seen that when a single line scan camera obtains point clouds for occlusion problems, point cloud contours are easily missing, adjusting the shooting angle affects production efficiency, and when multi-angle point cloud fusion is performed, the fusion result is easily affected by noise, which ultimately leads to large errors in the detection results.

[0003] In the binocular line scan camera measurement system, the occluded part is restored by another camera to restore the point cloud contour, and then the point cloud contours taken by two cameras at different angles are stitched together, which can effectively solve the occlusion problem, but the stitching process requires solving the position relationship between the cameras. The currently used binocular line scan camera solution can use point cloud contour registration and calibration fast feature extraction to solve the camera position relationship. The former is too complicated to calculate, and the calculation effect depends on the obvious characteristics of the photographed object. The latter has a simple calculation process, but requires the coplanarity of the laser bars of the two line scan cameras to obtain accurate results. The process of adjusting the coplanarity of the laser bars is relatively cumbersome. In short, neither can achieve efficient calibration results.

[0004] It can be seen that the camera calibration method in the prior art is difficult to achieve efficient calibration while solving the occlusion problem. Summary of the invention

[0005] The present invention provides a binocular line scan camera calibration method and system to solve the problem that the camera calibration method in the prior art is difficult to achieve efficient calibration while solving the occlusion problem.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] In a first aspect, the present invention provides a binocular line scan camera calibration method, which is applied to a binocular line scan camera calibration system, the system comprising a fixed platform, a first line scan camera and a second line scan camera mounted on the fixed platform, and a calibration block arranged in a common field of view of the first line scan camera and the second line scan camera, the method comprising:

[0008] S1: respectively collecting a first point cloud contour and a second point cloud contour of the surface of the calibration block by using a first line scan camera and a second line scan camera;

[0009] S2: Determine the main camera point cloud and the slave camera point cloud in the first cloud outline and the second point cloud outline;

[0010] S3: Determine a position transformation matrix between the first line scan camera and the second line scan camera;

[0011] S4: The main camera point cloud and the slave camera point cloud are merged according to the position transformation matrix to obtain the calibration result.

[0012] Optionally, a thin line is provided on the calibration block, and the calibration block is divided into N planes by the thin line, where N is a positive integer;

[0013] When the first line scan camera and the second line scan camera collect the first point cloud contour and the second point cloud contour of the calibration block surface respectively, the following setting conditions must be met:

[0014] When the laser strips of the first line scan camera and the second line scan camera are projected onto the same thin line, the two laser strips are constrained to be in the same plane.

[0015] Optionally, the first point cloud outline and the second point cloud outline each include two intersecting straight lines, and the first point cloud outline corresponds to a first coordinate system, and the second point cloud outline corresponds to a second coordinate system.

[0016] Optionally, S3 includes:

[0017] Construct a rotation matrix around the Y axis;

[0018] Calculate the baseline of the main camera;

[0019] Calculate the rotation line from the camera;

[0020] Calculate a first angle between the baseline and the X-axis, and calculate a second angle between the rotation line and the X-axis;

[0021] Determine an angle difference according to the first angle and the second angle;

[0022] Construct a translation matrix based on the angle difference;

[0023] The rotation matrix and the translation matrix are combined to form a position transformation matrix between the first line scan camera and the second line scan camera.

[0024] Optionally, calculate the baseline of the master camera and the rotation line of the slave camera, determined by:

[0025] If there are more points in the left contour of the master camera than in the right contour of the master camera, the baseline is the left contour of the master camera and the rotation line is the left contour of the slave camera; otherwise, the baseline is the right contour of the master camera and the rotation line is the right contour of the slave camera.

[0026] Optionally, a translation matrix is ​​constructed based on the angle difference, including:

[0027] After converting the angle difference into radians, calculate the rotation matrix around the Y axis, multiply the vertex by the rotation matrix around the Y axis and subtract it from the main vertex, and construct the translation matrix based on the difference.

[0028] Optionally, S4 includes: multiplying the position transformation matrix from the camera point cloud to transform it to the same coordinate system as the main camera, merging, resampling, and Gaussian smoothing the point cloud in the same coordinate system to obtain a calibrated point cloud.

[0029] In a second aspect, the present application provides a binocular line scan camera calibration system, the system comprising a fixed platform, a first line scan camera and a second line scan camera mounted on the fixed platform, a calibration block provided in a common field of view of the first line scan camera and the second line scan camera, a controller for controlling the first line scan camera and the second line scan camera, and a processor, the processor being configured to perform the following steps:

[0030] S1: respectively collecting a first point cloud contour and a second point cloud contour of the surface of the calibration block by using a first line scan camera and a second line scan camera;

[0031] S2: Determine the main camera point cloud and the slave camera point cloud in the first cloud outline and the second point cloud outline;

[0032] S3: Determine a position transformation matrix between the first line scan camera and the second line scan camera;

[0033] S4: The main camera point cloud and the slave camera point cloud are merged according to the position transformation matrix to obtain the calibration result.

[0034] Optionally, a thin line is provided on the calibration block, and the calibration block is divided into N planes by the thin line, where N is a positive integer.

[0035] Beneficial effects:

[0036] The binocular line scan camera calibration method provided by the present invention is applied to a binocular line scan camera calibration system. The method calculates the position transformation matrix through the binocular line scan camera calibration algorithm, which can solve the problem of incomplete point cloud contour due to occlusion. The position transformation matrix is ​​further used to fuse the first point cloud contour and the second point cloud contour. The direct calibration method is used to realize the fusion of point cloud contours at different angles, which can meet the real-time measurement of occluded scenes and improve the efficiency of measurement calibration.

[0037] In a further solution, a thin line is provided on the calibration block to ensure that the laser bars of the binocular line scan camera can be quickly coplanar, which can further improve the calibration speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a structural schematic diagram of a binocular line scan camera calibration system according to a preferred embodiment of the present invention;

[0039] Figure 2 The present invention is a flowchart of a binocular line scan camera calibration method according to a preferred embodiment of the present invention.

[0040] Reference numerals:

[0041] 1. Fixed platform; 2. First line scan camera; 3. Second line scan camera; 4. Calibration block; 5. Thin wire; 6. Controller; 7. Processor; 8. Display. DETAILED DESCRIPTION

[0042] The technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the usual meanings understood by persons with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as "one" or "one" do not indicate quantity restrictions, but indicate the existence of at least one. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship also changes accordingly.

[0044] During the assembly process of the entire vehicle production line, point cloud is missing due to occlusion. The existing monocular line scan cannot solve the occlusion problem well, and the current binocular line scan camera technology has the problems of cumbersome calibration and low efficiency, resulting in low vehicle inspection efficiency, complex inspection structure, and difficult inspection. In order to ensure the inspection efficiency and quality, this application proposes a binocular line scan camera calibration method and system.

[0045] It should be understood that the binocular line scan camera calibration method provided in this application is applied to a binocular line scan camera calibration system, such as Figure 1 As shown, the binocular line scan camera calibration system includes a fixed platform 1 for line scan cameras, a first line scan camera 2 and a second line scan camera 3 installed on the fixed platform 1, a calibration block 4 in the common field of view of the first line scan camera 2 and the second line scan camera 3, and a thin line 5 is provided on the calibration block. The system also includes a controller 6 connecting the first line scan camera 2 and the second line scan camera 3, a processor 7 connected to the controller 6, and a display 8 connected to the processor 7 for viewing the calibration results.

[0046] See also Figure 2 The present application provides a binocular line scan camera calibration method, which is applied to a binocular line scan camera calibration system. The system includes a fixed platform 1, a first line scan camera 2 and a second line scan camera 3 installed on the fixed platform 1, and a calibration block 4 arranged in a common field of view of the first line scan camera 2 and the second line scan camera 3. The method includes:

[0047] S1: respectively collecting a first point cloud contour and a second point cloud contour of the surface of the calibration block 4 by using the first line scan camera 2 and the second line scan camera 3 .

[0048] In this step, the lasers of the first line scan camera 2 and the second line scan camera 3 need to be projected onto the calibration module at the same time. During the acquisition process, the placement angle of the calibration block 4 is adjusted so that the two line scan cameras can capture the point cloud contours of the two inclined surfaces of the calibration block 4 as much as possible at the same time. Before acquisition, the installation positions of the first line scan camera 2 and the second line scan camera 3 are adjusted so that the laser stripes of the two are simultaneously projected onto the same thin line 5 of the calibration block 4, and the two laser stripes are coplanar. During the acquisition process, the two line scan cameras are controlled by the controller 6 to respectively acquire the single-frame point cloud contours of the surface of the high-precision calibration block 4, and the acquired contours include the two inclined surfaces and vertices of the calibration block 4.

[0049] S2: Determine the main camera point cloud and the slave camera point cloud in the first point cloud outline and the second point cloud outline.

[0050] In one example, the first point cloud outline can be set as the master camera point cloud, and the second point cloud outline can be set as the slave camera point cloud. The camera corresponding to the master camera point cloud is the master camera, and the camera corresponding to the slave camera point cloud is the slave camera.

[0051] S3: Determine the position transformation matrix between the first line scan camera 2 and the second line scan camera 3.

[0052] S4: The main camera point cloud and the slave camera point cloud are merged according to the position transformation matrix to obtain the calibration result.

[0053] The above-mentioned binocular line scan camera calibration method calculates the position transformation matrix through the binocular line scan camera calibration algorithm, which can solve the problem of incomplete point cloud contour due to occlusion. The position transformation matrix is ​​further used to fuse the first point cloud contour and the second point cloud contour. The direct calibration method is used to realize the fusion of point cloud contours at different angles, which can meet the real-time measurement of occluded scenes and improve the efficiency of measurement calibration.

[0054] Below, taking a complete example, the steps of the above binocular line scan camera calibration method are described in detail as follows:

[0055] Step 1: Install the first line scan camera 2 and the second line scan camera 3 on the fixed platform 1, connect the controller 6 to the relevant ports of the line scan cameras, turn on the two line scan cameras, project the laser bars of the two cameras onto the high-precision calibration block 4, adjust the placement angle of the calibration block 4, and make the two line scan cameras capture the point cloud contours of the two inclined surfaces of the calibration block 4 as much as possible at the same time;

[0056] Step 2: After completing step 1, adjust the installation position of the line scan camera so that the two laser stripes are simultaneously projected on the same fine line 5 of the calibration block 4, and the two laser stripes are coplanar. In this way, fine straight line processing on the calibration block 4 can quickly assist the staff to achieve the coplanarity of the line scan camera laser stripes, thereby improving production efficiency.

[0057] Step 3: After completing step 2, the controller 6 controls the two line scan cameras to respectively collect single-frame point cloud contours of the surface of the high-precision calibration block 4. The contours collected include two inclined surfaces and vertices of the calibration block 4. The point cloud contours are expressed as: two intersecting straight lines, and the point method ax+by+cz+d=0 is used to represent the straight lines, a, b, c, d represent the straight line parameters, and xyz represents the three axes; the direction vector is n, n=(a, c, b), and the two frames of point cloud contours belong to different line scan camera coordinate systems;

[0058] Step 4: After completing step 3, set one camera coordinate system point cloud as the main camera point cloud and the other as the slave camera point cloud. Assume that a point on the main camera is P master =[x master y master z master ] T , [] T is the matrix transpose, from a point on the camera to P slave＝ [x slave y slave z slave ] T , the conversion relationship between the two is described as:

[0059] P master =R×P slave +t;

[0060] Where R represents point P slave To point P master The rotation matrix of point P slave To point P master The translation matrix of

[0061] in:

[0062]

[0063] t=[t x ,0,t z] T ;

[0064] In the formula, t x Indicates point P slave After left multiplication with the rotation matrix R and P master The distance between them in the x direction, t z Indicates point P slave After left multiplication with the rotation matrix R and P master is the distance in the z direction, and θ represents the angle between the contour line and the x-axis.

[0065] Since the coplanarity of the laser bars ensures that the contours of the two point clouds are coplanar, there is no difference in the Y direction between the two contours, so only a rotation matrix around the Y axis is constructed. The rotation angle θ around the Y axis can be determined by the difference between the baseline, the rotation line and the X axis, and t can be determined by P slave Multiply R and P on the left master Determined by subtraction; wherein, the X-axis direction refers to the horizontal direction in the point cloud contour coordinate system, and the Y-axis direction refers to the vertical direction in the point cloud contour coordinate system.

[0066] To calculate the baseline, the first line scan camera 2 is selected as the main camera, and the straight line fitting algorithm (Random Sample Consensus, RANSAC) is used to calculate the two straight line normal vectors n in the first point cloud contour. master-l 、n master-r , then calculate the coordinates of the intersection of the two straight lines as the main vertex P master-cross , which corresponds to the intersection of the two inclined planes on the calibration block 4, and is divided into the "left" straight line l with the main vertex coordinate as the reference master-l and the "right" straight line l master-r ;

[0067] To further calculate the rotation line, the second line scan camera 3 is selected as the slave camera, and the RANSAC straight line fitting algorithm is used to calculate the two straight line normal vectors n in the point cloud contour. slave-l 、n slave-r , calculate the coordinates of the intersection of the two straight lines according to the parameters as the starting point of vertex P slave-cross , to keep consistent with the "left" and "right" of the main camera, select the main camera point P master-cross Divide into "left" straight line l for reference slave-l and the "right" straight line l slave-r ;

[0068] Further need to be in the main camera l master-l , l master-r Determine the baseline l between two straight lines base , from the camera l slave-l , l slave-r Determine the rotation line l between two straight lines rotate, the rule is as follows: if there are more points in the left contour of the master camera than in the right contour of the master camera, then the baseline is the left contour of the master camera and the rotation line is the left contour of the slave camera; otherwise, the baseline is the right contour of the master camera and the rotation line is the right contour of the slave camera. This rule satisfies the following relationship:

[0069]

[0070] Among them, sizeof() represents the number of contour points, l base Indicates the baseline, l rotate represents the rotation line, l master-l Indicates the left outline of the main camera, l master-r Indicates the right side of the main camera, l slave-l Indicates the outline from the left side of the camera, l slave-r represents the outline from the right side of the camera. In other words, when l master-l There are more points in l master-r , the baseline is l master-l , the rotation line is l slave-l ; On the contrary, the baseline is l master-r , the rotation line is l slave-r .

[0071] Further calculation of l base , l rotate The angle θ between them and the X axis base ,θ rotate The difference between the two is θ, and the straight line l rotate Can be rotated around the Y axis by angle θ to achieve the same base parallel;

[0072] The angle difference θ is further converted into radians and the rotation matrix R around the Y axis is calculated. slave-cross After left multiplication by matrix R, add master-cross Subtract the difference to form the translation matrix t. The rotation matrix and the translation matrix are combined to form the position transformation matrix between the two cameras. The transformation matrix is ​​the calibration result.

[0073] Step 5: Multiply the point cloud contour in the camera coordinate system by the transformation matrix and transform it to the same coordinate system as the main camera. Merge, resample, and Gaussian smooth the point clouds in the same coordinate system to obtain the final display point cloud.

[0074] Step 6: After step 5 is completed, the calibration results are output in text format and the final point cloud results are displayed. Users can judge the calibration results based on the overlap and the contour of the merged point cloud.

[0075] The present application also provides a binocular line scan camera calibration system, the system comprising a fixed platform 1, a first line scan camera 2 and a second line scan camera 3 mounted on the fixed platform 1, a calibration block 4 arranged in a common field of view of the first line scan camera 2 and the second line scan camera 3, a controller 6 and a processor 7 for controlling the first line scan camera 2 and the second line scan camera 3, the processor 7 being used to perform the following steps:

[0076] S1: respectively collecting a first point cloud contour and a second point cloud contour of a surface of a calibration block 4 by means of a first line scan camera 2 and a second line scan camera 3;

[0077] S2: Determine the main camera point cloud and the slave camera point cloud in the first cloud outline and the second point cloud outline;

[0078] S3: Determine a position transformation matrix between the first line scan camera 2 and the second line scan camera 3;

[0079] S4: The main camera point cloud and the slave camera point cloud are merged according to the position transformation matrix to obtain the calibration result.

[0080] Specifically, a fine line 5 is provided on the calibration block 4, and the calibration block 4 is divided into N planes by the fine line 5, where N is a positive integer. The fine line processing can quickly assist the staff to realize the coplanarity of the laser strips of the line scan camera, thereby improving production efficiency.

[0081] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A binocular line scan camera calibration method, applied to a binocular line scan camera calibration system, characterized in that: The system comprises a fixed platform, a first line scan camera and a second line scan camera mounted on the fixed platform, and a calibration block arranged in a common field of view of the first line scan camera and the second line scan camera. The method comprises: S1: respectively collecting a first point cloud profile and a second point cloud profile of the surface of the calibration block by using the first line scan camera and the second line scan camera; S2: Determine a master camera point cloud and a slave camera point cloud in the first point cloud outline and the second point cloud outline; S3: Determine a position transformation matrix between the first line scan camera and the second line scan camera; S4: merging the master camera point cloud and the slave camera point cloud according to the position transformation matrix to obtain a calibration result.

2. The binocular line scan camera calibration method according to claim 1, characterized in that: The calibration block is provided with a thin line, and the calibration block is divided into N planes by the thin line, where N is a positive integer; When the first line scan camera and the second line scan camera respectively collect the first point cloud contour and the second point cloud contour of the surface of the calibration block, the following setting conditions must be met: When the laser bars of the first line scan camera and the second line scan camera are projected onto the same thin line, the two laser bars are constrained to be in the same plane.

3. The binocular line scan camera calibration method according to claim 1, characterized in that: The first point cloud outline and the second point cloud outline each include two intersecting straight lines, and the first point cloud outline corresponds to a first coordinate system, and the second point cloud outline corresponds to a second coordinate system.

4. The binocular line scan camera calibration method according to claim 1, characterized in that: The S3 includes: Construct a rotation matrix around the Y axis; Calculate the baseline of the main camera; Calculate the rotation line from the camera; Calculating a first angle between the baseline and the X-axis, and calculating a second angle between the rotation line and the X-axis; determining an angle difference according to the first angle and the second angle; Constructing a translation matrix according to the angle difference; The rotation matrix and the translation matrix are combined to form a position transformation matrix between the first line scan camera and the second line scan camera.

5. The binocular line scan camera calibration method according to claim 4, characterized in that: The calculation of the baseline of the main camera and the rotation line of the slave camera are determined by: If there are more points in the left contour of the master camera than in the right contour of the master camera, the baseline is the left contour of the master camera and the rotation line is the left contour of the slave camera; otherwise, the baseline is the right contour of the master camera and the rotation line is the right contour of the slave camera.

6. The binocular line scan camera calibration method according to claim 4, characterized in that: The constructing a translation matrix according to the angle difference comprises: After converting the angle difference into radians, calculate the rotation matrix around the Y axis, multiply the vertex by the rotation matrix around the Y axis and subtract it from the main vertex, and construct the translation matrix based on the difference.

7. The binocular line scan camera calibration method according to claim 1, characterized in that: The S4 includes: multiplying the position transformation matrix from the camera point cloud to the same coordinate system as the main camera, merging, resampling, and Gaussian smoothing the point cloud in the same coordinate system to obtain a calibrated point cloud.

8. A binocular line scan camera calibration system, characterized in that: The system includes a fixed platform, a first line scan camera and a second line scan camera mounted on the fixed platform, a calibration block provided in a common field of view of the first line scan camera and the second line scan camera, a controller for controlling the first line scan camera and the second line scan camera, and a processor, wherein the processor is used to perform the following steps: S1: respectively collecting a first point cloud profile and a second point cloud profile of the surface of the calibration block by using the first line scan camera and the second line scan camera; S2: Determine a master camera point cloud and a slave camera point cloud in the first cloud outline and the second point cloud outline; S3: Determine a position transformation matrix between the first line scan camera and the second line scan camera; S4: merging the master camera point cloud and the slave camera point cloud according to the position transformation matrix to obtain a calibration result.

9. The binocular line scan camera calibration system according to claim 8, characterized in that: The calibration block is provided with a thin line, and the calibration block is divided into N planes by the thin line, where N is a positive integer.

Images