Laterally-mounted laser triangulation measurement device based on double cameras with different focal lengths and measurement method

Abstract

According to the side-mounted laser triangulation device and method based on the double cameras with different focal lengths, two overlapped material pictures are obtained at the same time, the pictures are spliced in a pixel level, and the three-dimensional attributes of materials are obtained through laser triangulation; the problem of maintenance caused by traditional installation above the belt conveyor and the problem that the whole surface of the conveyor belt cannot be detected are solved.

Description

Technical Field

[0001] The invention relates to a laser triangulation measuring device and a measuring method based on dual cameras with different focal lengths installed on the side, belonging to the technical field of image processing. Background Art

[0002] In the field of image processing technology, the extraction and visualization of laser lines are crucial for multiple application scenarios, including 3D reconstruction, virtual reality, industrial measurement, etc. Belt conveyors used in ports and mines often use laser triangulation with industrial cameras to monitor the conveyor belts. Traditional laser triangulation methods are mainly based on a single camera. However, belt conveyors used in ports, mines and other scenes are often wide. At traditional heights, most industrial cameras on the market cannot take full laser images of the conveyor belts. If the height is increased, the details of the material transmission cannot be clearly obtained, resulting in deviations in the calculation of the material volume, and the increased height is not conducive to the inspection and maintenance of industrial cameras.

[0003] Adding more industrial cameras directly above the conveyor belt to cover the entire conveyor belt in the width direction will make future maintenance difficult, requiring the machine to be stopped for maintenance. Installing industrial cameras on the side of the conveyor belt will make maintenance easier, but the laser triangulation will have inconsistent accuracy in both near and far measurements due to oblique projection. Summary of the invention

[0004] The object of the present invention is to provide a side-mounted laser triangulation measurement device and a measurement method based on dual cameras with different focal lengths.

[0005] The present invention extracts the laser line from the grayscale image collected by the camera equipped with a filter, projects the laser line into the color image collected by the camera without a filter using the external reference between the binocular cameras, and refines the laser line through the rich texture information of the color image to achieve accurate laser line extraction and visualization. This solves the maintenance problems caused by the traditional installation above the belt conveyor and the problem of being unable to detect the entire surface of the conveyor belt.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is:

[0007] A laser triangulation device based on dual cameras with different focal lengths mounted on the side, characterized in that the image projected on the target surface is collected by the camera, and the three-dimensional position of the target is determined by triangulation using the position relationship between the laser in the image and the camera laser. The laser triangulation device includes a short-focus industrial camera c1, a long-focus industrial camera c2, and a laser transmitter.

[0008] The laser transmitter emits laser light, and the frequency of the laser light can be adjusted; the short-focus industrial camera c1 uses a short-focus camera with a focal length less than 50mm; the long-focus industrial camera c2 uses a long-focus camera with a focal length greater than 50mm; there is a common viewing area in the camera images of the short-focus industrial camera c1 and the long-focus industrial camera c2, and both can observe the laser line shining on the surface of the object.

[0009] Furthermore, a laser triangulation method based on a side-mounted dual camera with different focal lengths includes the following contents:

[0010] The steps of the detection method are as follows:

[0011] First, the laser generator emits laser light to illuminate the contour line on the surface of the target to be detected;

[0012] Secondly, calibration is performed to calibrate the external parameters between the short-focus industrial camera c1 and the long-focus industrial camera c2, calibrate the internal parameters of the short-focus industrial camera c1 and the long-focus industrial camera c2, and calibrate the plane equations in the coordinate system of the short-focus industrial camera c1 and the long-focus industrial camera c2;

[0013] Then, the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 is calculated, and the laser line in the image captured by the long-focus industrial camera c2 is transformed into the image of the short-focus industrial camera c1 using the transformation matrix, and then fused;

[0014] Finally, the fused short-focus industrial camera C1 image and the laser plane equation are used to calculate the three-dimensional position of the laser.

[0015] Further, the process of calibrating the plane equations in the coordinate systems of the short-focus industrial camera c1 and the long-focus industrial camera c2 includes:

[0016] Calibrate the rotation matrix R and translation matrix t between the short-focus industrial camera c1 and the long-focus industrial camera c2. The internal parameter of the short-focus industrial camera c1 is K c1 , the internal reference of the telephoto industrial camera c2 is K c2 ; In the short-focus industrial camera c1 coordinate system, the normal vector of the laser plane L1 is The vertical distance from the origin O to the laser plane is d; under the telephoto industrial camera c2, the equation of the laser plane is L2.

[0017] The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C1 and T C1 ; The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C2 and T C2 ; Point X under the laser plane w , the corresponding point in the c1 coordinate system of the short-focus industrial camera is P c1, the corresponding point in the c2 coordinate system of the telephoto industrial camera is P c2 .

[0018] Further, the process of calculating the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 includes: first obtaining the matrix expression of the laser plane L1, and then obtaining the transformation matrix of the short-focus industrial camera c1 according to the matrix expression of the laser plane L1;

[0019] The process of solving the matrix expression of the laser plane L1 includes:

[0020] The coordinate system of the short-focus industrial camera c1 is taken as the reference system, that is, the camera coordinate system of the short-focus industrial camera c1 is the world coordinate system, then R C1 =E,T c1 =0

[0021] Point X on the laser plane in the world coordinate system w With P t1 (x t1 ,y t1 ,1) and P t2 (x t2 ,y t2 ,1)Satisfy the transformation relationship

[0022]

[0023] Set the laser plane point X w Expressed in non-homogeneous form:

[0024]

[0025]

[0026] In the short-focus industrial camera c1 coordinate system, the laser plane L1 can be expressed as:

[0027]

[0028] in is the normal vector of the laser plane in the short-focus industrial camera c1 coordinate system, d represents the vertical distance from the plane to the camera A, is a point on the plane in the camera c1 coordinate system. and are two coefficients.

[0029] Further, the process of obtaining the transformation matrix of c1 according to the matrix expression of the laser plane L1 includes:

[0030] By mathematically transforming the matrix expression of the laser plane L1, the following quantitative relationship is obtained:

[0031]

[0032] Because of the optical properties is the non-homogeneous coordinate of the laser in the camera coordinate system. Substituting (a), (b) and (c) mathematically, we get:

[0033]

[0034] in

[0035]

[0036] Because R c1 =E, So R c2 =R,P c2 =T, and

[0037] So the final homography matrix is:

[0038]

[0039] Furthermore, the process of transforming the laser line in the image captured by the long-focus industrial camera c2 into the image of the short-focus industrial camera c1 and fusing the image using the transformation matrix includes:

[0040] The laser point P in the telephoto industrial camera t2 Projected to point P in the c1 coordinate system of the short-focus industrial camera t1 The conversion formula P t1 =H -1 P t2 ; Average the laser points in the overlapping area; If the projection point P t1 and the laser point P in the short-focus industrial camera c1 c1 The x-axis coordinates of are equal, then the coordinates of the fused point are P' c1 =(P t2 +P c1 ) / 2.

[0041] Furthermore, the process of calculating the three-dimensional position of the laser using the fused short-focus industrial camera c1 image and the laser plane equation includes:

[0042] According to the geometric constraints of the three-dimensional points on the laser line, the equations are listed as follows:

[0043]

[0044] Where A, B, C, D are the parameters of the laser plane L1, x B ,y B 、z Bis the coordinate of a point in three-dimensional space, u B 、v B is a point in the image of the short-focus industrial camera c1 after fusion. After solving the above equation, we can get

[0045]

[0046] in They are The first and second rows.

[0047] The present invention has the following advantages:

[0048] 1) The present invention installs dual cameras with different focal lengths on the side of the belt. The cameras with different focal lengths respectively measure different near and far areas on the belt to cover the entire belt, and the acquisition results of the cameras with different focal lengths are fused and spliced, so as to solve the problem of laser triangulation equipment covering a wide belt without reducing the measurement accuracy, and facilitate construction and maintenance.

[0049] 2) The present invention improves the problems of incomplete measurement, reduced measurement accuracy, difficult construction and maintenance, etc. in the application of laser triangulation in ultra-wide belts. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a diagram of the light intensity type weak measurement high-precision area array optical rotation detection system of the present invention.

[0051] Figure 2 It is the algorithm flow chart of the present invention.

[0052] Figure 3 It is the projection of a point on the plane of the present invention to the camera.

[0053] Figure 4 This is the laser point transformation and fusion of the present invention. DETAILED DESCRIPTION

[0054] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, a side-mounted laser triangulation measurement device based on dual cameras with different focal lengths includes a short-focus industrial camera c1, a long-focus industrial camera c2, a laser transmitter, and a fixed column; wherein the laser transmitter, the short-focus industrial camera c1, and the long-focus industrial camera c2 are mounted on the fixed column;

[0056] The laser transmitter emits laser light, and the frequency of the laser light can be adjusted; the short-focus industrial camera c1 uses a short-focus camera with a focal length less than 50mm; the long-focus industrial camera c2 uses a long-focus camera with a focal length greater than 50mm; there is a common viewing area in the camera images of the short-focus industrial camera c1 and the long-focus industrial camera c2, and both can observe the laser line shining on the surface of the object.

[0057] In summary, if Figure 2 A laser triangulation method based on a side-mounted dual camera with different focal lengths is shown, comprising the following contents:

[0058] The steps of the detection method are as follows:

[0059] First, the laser generator emits laser light to illuminate the contour line on the surface of the target to be detected;

[0060] Secondly, calibration is performed to calibrate the external parameters between the short-focus industrial camera c1 and the long-focus industrial camera c2, calibrate the internal parameters of the short-focus industrial camera c1 and the long-focus industrial camera c2, and calibrate the plane equations in the coordinate system of the short-focus industrial camera c1 and the long-focus industrial camera c2;

[0061] Then, the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 is calculated, and the laser line in the image captured by the long-focus industrial camera c2 is transformed into the image of the short-focus industrial camera c1 using the transformation matrix, and then fused;

[0062] Finally, the fused short-focus industrial camera C1 image and the laser plane equation are used to calculate the three-dimensional position of the laser.

[0063] Further, the process of calibrating the plane equations in the coordinate systems of the short-focus industrial camera c1 and the long-focus industrial camera c2 includes:

[0064] Calibrate the rotation matrix R and translation matrix t between the short-focus industrial camera c1 and the long-focus industrial camera c2. The internal parameter of the short-focus industrial camera c1 is K c1 , the internal reference of the telephoto industrial camera c2 is K c2 ; In the short-focus industrial camera c1 coordinate system, the normal vector of the laser plane L1 is The vertical distance from the origin O to the laser plane is d; under the telephoto industrial camera c2, the equation of the laser plane is L2.

[0065] The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C1 and T C1 ; The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C2 and T C2 .

[0066] Point X under the laser planew , the corresponding point in the c1 coordinate system of the short-focus industrial camera is P c1 , the corresponding point in the c2 coordinate system of the telephoto industrial camera is P c2 .

[0067] Further, the process of calculating the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 includes: first obtaining the matrix expression of the laser plane L1, and then obtaining the transformation matrix of c1 according to the matrix expression of the laser plane L1;

[0068] The process of solving the matrix expression of the laser plane L1 includes:

[0069] The coordinate system of the short-focus industrial camera c1 is taken as the reference system, that is, the camera coordinate system of the short-focus industrial camera c1 is the world coordinate system, then R C1 =E,T c1 =0

[0070] Point X on the laser plane in the world coordinate system w With P t1 (x t1 ,y t1 ,1) and P t2 (x t2 ,y t2 ,1)Satisfy the transformation relationship

[0071]

[0072] Set the laser plane point X w Expressed in non-homogeneous form:

[0073]

[0074] In the short-focus industrial camera c1 coordinate system, the laser plane L1 can be expressed as:

[0075]

[0076] in is the normal vector of the laser plane in the short-focus industrial camera c1 coordinate system, d represents the vertical distance from the plane to the camera A, It is a point on the plane of the short-focus industrial camera c1 coordinate system. and are two coefficients.

[0077] Further, the process of obtaining the transformation matrix of c1 according to the matrix expression of the laser plane L1 includes:

[0078] By mathematically transforming the matrix expression of the laser plane L1, the following quantitative relationship is obtained:

[0079]

[0080] Because of the optical properties is the non-homogeneous coordinate of the laser in the camera coordinate system. Substituting (a), (b) and (c) mathematically, we get:

[0081]

[0082] in

[0083]

[0084] Because R c1 =E, So R c2 =R,T c2 =T, and

[0085] So the final homography matrix is:

[0086]

[0087] Furthermore, the process of transforming the laser line in the image captured by the long-focus industrial camera c2 into the image of the short-focus industrial camera c1 and fusing the image using the transformation matrix includes:

[0088] The laser point P in the telephoto industrial camera c2 t2 Projected to point P in the c1 coordinate system of the short-focus industrial camera t1 The conversion formula P t1 =H -1 P t2 ; Average the laser points in the overlapping area; If the projection point P t1 and the laser point P in the short-focus industrial camera c1 c1 The x-axis coordinates of are equal, then the coordinates of the fused point are P' c1 =(P t2 +P c1 ) / 2.

[0089] Furthermore, the process of calculating the three-dimensional position of the laser using the fused short-focus industrial camera c1 image and the laser plane equation includes:

[0090] According to the geometric constraints of the three-dimensional points on the laser line, the equations are listed as follows:

[0091]

[0092] Where A, B, C, D are the parameters of the laser plane L1, x B ,y B 、z Bis the coordinate of a point in three-dimensional space, u B 、v B is a point in the image of the short-focus industrial camera c1 after fusion. After solving the above equation, we can get

[0093]

[0094] in They are The first and second rows.

[0095] The present invention has the following advantages:

[0096] 1) The present invention installs dual cameras with different focal lengths on the side of the belt. The cameras with different focal lengths respectively measure different near and far areas on the belt to cover the entire belt, and the acquisition results of the cameras with different focal lengths are fused and spliced, so as to solve the problem of laser triangulation equipment covering a wide belt without reducing the measurement accuracy, and facilitate construction and maintenance.

[0097] 2) The present invention improves the problems of incomplete measurement, reduced measurement accuracy, difficult construction and maintenance, etc. in the application of laser triangulation in ultra-wide belts.

[0098] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A side-mounted laser triangulation device based on dual cameras with different focal lengths, characterized in that: The image projected on the target surface is collected by the camera, and the three-dimensional position of the target is determined by triangulation using the position relationship between the laser in the image and the camera laser.

2. A side-mounted laser triangulation measurement device based on dual cameras with different focal lengths according to claim 1, characterized in that: The laser triangulation measuring device comprises a short-focus industrial camera c1, a long-focus industrial camera c2, and a laser transmitter.

3. A side-mounted laser triangulation device based on dual cameras with different focal lengths according to claim 2, characterized in that: The laser transmitter emits laser light. The short-focus industrial camera c1 uses a short-focus camera with a focal length less than 50mm; the long-focus industrial camera c2 uses a long-focus camera with a focal length greater than 50mm. There is a common viewing area in the camera images of the short-focus industrial camera c1 and the long-focus industrial camera c2, and both can observe the laser line irradiated on the surface of the object.

4. A laser triangulation method based on dual cameras with different focal lengths mounted on the side, implemented according to a laser triangulation device based on dual cameras with different focal lengths mounted on the side according to any one of claims 1 to 3, characterized in that: The steps of the detection method include the following: First, the laser generator emits laser light to illuminate the contour line on the surface of the target to be detected. According to the set laser color and brightness, the area with a specific color and a larger brightness value in the image is extracted as the laser line in the image; Secondly, calibration is performed to calibrate the external parameters between the short-focus industrial camera c1 and the long-focus industrial camera c2, calibrate the internal parameters of the short-focus industrial camera c1 and the long-focus industrial camera c2, and calibrate the plane equations in the coordinate system of the short-focus industrial camera c1 and the long-focus industrial camera c2; Then, the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 is calculated, and the laser line in the image captured by the long-focus industrial camera c2 is transformed into the image of the short-focus industrial camera c1 using the transformation matrix, and then fused; Finally, the fused short-focus industrial camera C1 image and the laser plane equation are used to calculate the three-dimensional position of the laser.

5. A laser triangulation method based on dual cameras with different focal lengths mounted on the side according to claim 4, wherein: The process of calibrating the plane equations in the coordinate systems of the short-focus industrial camera c1 and the long-focus industrial camera c2 includes: Calibrate the rotation matrix R and translation matrix t between the short-focus industrial camera c1 and the long-focus industrial camera c2. The internal parameter of the short-focus industrial camera c1 is K c1 , the internal reference of the telephoto industrial camera c2 is K c2 ; In the short-focus industrial camera c1 coordinate system, the normal vector of the laser plane L1 is The vertical distance from the origin O to the laser plane is d; under the industrial camera c2, the equation of the laser plane is L2; The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C1 and T C1 ; The rotation and translation matrix of the short-focus industrial camera c1 coordinate system and the world coordinate system is R C2 and T C2 ; Point X under the laser plane w , the corresponding point in the c1 coordinate system is P c1 , the corresponding point in the c2 coordinate system is P c2 .

6. A laser triangulation method based on dual cameras with different focal lengths mounted on the side according to claim 5, wherein: The process of calculating the transformation matrix from the laser line in the image captured by the long-focus industrial camera c2 to the short-focus industrial camera c1 includes: first obtaining the matrix expression of the laser plane L1, and then obtaining the transformation matrix of the short-focus industrial camera c1 according to the matrix expression of the laser plane L1; The process of solving the matrix expression of the laser plane L1 includes: The coordinate system of the short-focus industrial camera c1 is taken as the reference system, that is, the camera coordinate system of the short-focus industrial camera c1 is the world coordinate system, then R C1 =E,T c1 =0; Point X on the laser plane in the world coordinate system w With P t1 (x t1 ,y t1 ,1) and P t2 (x t2 ,y t2 ,1)Satisfy the transformation relationship: Set the laser plane point X w Expressed in non-homogeneous form: In the short-focus industrial camera c1 coordinate system, the laser plane L1 is expressed as: in is the normal vector of the laser plane in the short-focus camera c1 coordinate system, d represents the vertical distance from the plane to camera A, It is a point on the plane of the short-focus industrial camera c1 coordinate system.

7. A laser triangulation method based on dual cameras with different focal lengths mounted on the side according to claim 6, wherein: The process of obtaining the transformation matrix of the short-focus camera c1 according to the matrix expression of the laser plane L1 includes: By mathematically transforming the matrix expression of the laser plane L1, the following quantitative relationship is obtained: in, and are the coefficients of laser plane L1 and laser plane L2 respectively; is the non-homogeneous coordinate of the laser in the camera coordinate system. Substituting (a), (b) and (c) mathematically, we get: in Because R c1 =E, So R c2 =R,T c2 =T, and So the final homography matrix is:

8. A laser triangulation method based on dual cameras with different focal lengths mounted on the side according to claim 7, wherein: The process of using the transformation matrix to transform the laser line in the image captured by the long-focus industrial camera c2 into the image of the short-focus industrial camera c1 and then fusing it includes: The laser point P in the telephoto industrial camera t2 Projected to point P in the c1 coordinate system of the short-focus industrial camera t1 The conversion formula P t1 =H -1 P t2 ; Average the laser points in the overlapping area; If the projection point P t1 and the laser point P in the short-focus industrial camera c1 c1 The x-axis coordinates of are equal, then the coordinates of the fused point are P' c1 =(P t2 +P c1 ) / 2.

9. A laser triangulation method based on dual cameras with different focal lengths mounted on the side according to claim 8, wherein: The process of calculating the three-dimensional position of the laser using the fused short-focus industrial camera C1 image and the laser plane equation includes: According to the geometric constraints of the three-dimensional points on the laser line, the equations are listed as follows: Where A, B, C, D are the parameters of the laser plane L1, x B ,y B 、z B is the coordinate of a point in three-dimensional space, u B 、v B is a point in the image of the short-focus industrial camera c1 after fusion; after solving the above equation, we get: in They are The first and second rows.

Images