Mobile crane tower assembling docking monitoring system based on machine vision technology

Abstract

The invention relates to the technical field of cranes, in particular to a mobile crane tower assembly docking monitoring system based on a machine vision technology. The method comprises the following steps: calibrating an installation position of a binocular camera, and obtaining internal parameters and external parameters of the camera so as to facilitate subsequent image processing; stereo correction is carried out, and image distortion caused by position difference between the two cameras is eliminated; stereo matching is used for determining the corresponding relation between the images, shot by the two cameras, of the same scene at different visual angles; and recovering the three-dimensional structure information of the scene, namely the depth map, according to the corresponding relation. According to the method, data visualization of the tower section butt joint progress and position is achieved through the machine vision technology, a driver can better control the hoisted tower section conveniently, and the tower assembling construction safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cranes, and in particular to a mobile crane tower assembly and docking monitoring system based on machine vision technology. Background Art

[0002] In the field of crane technology, in order to reduce labor intensity, improve work efficiency and ensure worker safety;

[0003] In the existing technology, mobile cranes are used to replace traditional tower assembly, which reduces the workload of pole transportation, assembly and dismantling. At the same time, mobile cranes have advanced safety control systems and load and position limit alarm systems, which effectively reduce the safety risks of operations such as bad weather and overload, reduce the frequency of high-altitude operations, and greatly improve the efficiency of tower assembly construction;

[0004] However, in actual use, mobile cranes have high requirements for the road conditions at the tower site and the actual terrain of the operation site. Under the condition of limited line corridor resources, the site applicability of mobile cranes is limited;

[0005] In view of the above situation, the current method of disassembly and assembly or overall assembly is adopted, and cranes of different tonnages are selected on site according to the tower parameters and terrain conditions. However, mobile cranes have high requirements on the road conditions for access to the tower and the actual terrain of the operation site. Under the condition of limited line corridor resources, the site applicability of mobile cranes is limited. Summary of the invention

[0006] The present invention provides a mobile crane tower assembly docking monitoring system based on machine vision technology, which can overcome certain defects of the prior art.

[0007] A mobile crane tower assembly docking monitoring system based on machine vision technology according to the present invention comprises:

[0008] Step S1, calibrate the installation position of the binocular camera to obtain the internal and external parameters of the camera for subsequent image processing;

[0009] Step S2: stereo correction to eliminate image distortion caused by the position difference between the two cameras;

[0010] Step S3: stereo matching, which is a process for determining the correspondence between images of the same scene captured by two cameras from different perspectives;

[0011] Step S4: According to the above correspondence, restore the three-dimensional structure information of the scene, that is, the depth map.

[0012] Preferably, in step S1, the camera calibration is completed according to the following steps:

[0013] Step S1.1: Calibrate the camera without considering the radial distortion, tangential distortion and thin prism distortion of the camera: DLT calibration method is used. The mathematical model of DLT calibration method is:

[0014]

[0015] Step S1.2: Arrange and eliminate z c , we get two about m ij The linear equation is:

[0016]

[0017] Step S1.3: Write equation (3-7) in matrix form:

[0018] Lm=0 (3-8)

[0019] In the above formula, L is a matrix with 2n rows and 12 columns, and n is the number of calibration points involved in the calculation;

[0020] Step S1.4: From (3-8), we can see that if m ij Assuming the pixel coordinates and world coordinates of the calibration points as unknowns, the known parameters are the parameters. Then, there are 12 unknowns in equation (3-8). From equation (3-7), it can be seen that one calibration point can correspond to two linear equations. Therefore, if we want to solve the 12 unknowns in equation (3-8), we need n calibration points (n≥6) to obtain the overdetermined equations about the 12 unknowns. Then, we can use the linear least squares method to solve equation (3-8) and solve the transformation coefficient m. ij .

[0021] Preferably, in step S2, stereo correction is performed according to the following steps:

[0022] Step S2.1: Ray o can be obtained through camera I and camera II respectively a1 p and o a1 p's spatial information, and obtains ray o a1 p and o a1 When solving the coordinates of point P, find the relationship between the world coordinates of the spatial point P and the image coordinates of the corresponding projection points p1 and p2:

[0023]

[0024] Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of the projection points p1 and p2 respectively, (x w ,y w ,z w,1) is the world homogeneous coordinate of point P; M1 and M2 are the projection matrices of the two cameras; f1 and f2 are the focal lengths of camera I and camera II respectively;

[0025] Step S2.2: Eliminate f1 and f2 to obtain w ,y w ,z w The linear equations

[0026]

[0027] Preferably, in step S3, stereo matching is completed according to the following steps:

[0028] Step S3.1: The spatial three-dimensional coordinates of the spatial point P can be obtained by combining the above equations. Considering the interference of data noise, it is necessary to solve the equations according to the least squares method and rewrite the above equation in the form of a matrix:

[0029]

[0030] The above formula can be simplified to KX = U, where X is the world coordinate of the desired spatial point, and K and U are known vectors that can be obtained by camera calibration. The least squares method can be used to solve it:

[0031]

[0032] Preferably, in step S4, the observed three-dimensional structure information is restored according to the following steps:

[0033] According to the above formula, the spatial three-dimensional coordinates of each point on the photo are obtained from the photos taken by the binocular vision camera and the internal and external parameters of the camera. All points on the object under test can find corresponding matching points on the image section to restore the three-dimensional geometric shape of the object under test. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is the basic model of visual imaging. DETAILED DESCRIPTION

[0035] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the embodiments. It should be understood that the embodiments are only for explaining the present invention and are not intended to limit it.

[0036] Example 1

[0037] This embodiment provides a mobile crane tower assembly docking monitoring system based on machine vision technology, which can overcome certain defects of the prior art.

[0038] According to this embodiment, a mobile crane tower assembly docking monitoring system based on machine vision technology includes:

[0039] Step S1, calibrate the installation position of the binocular camera to obtain the internal and external parameters of the camera for subsequent image processing;

[0040] Step S2: stereo correction to eliminate image distortion caused by the position difference between the two cameras;

[0041] Step S3: stereo matching, which is a process for determining the correspondence between images of the same scene captured by two cameras from different perspectives;

[0042] Step S4: According to the above correspondence, restore the three-dimensional structure information of the scene, that is, the depth map.

[0043] In step S1, the camera calibration is completed according to the following steps:

[0044] Step S1.1: Calibrate the camera without considering the radial distortion, tangential distortion and thin prism distortion of the camera: DLT calibration method is used. The mathematical model of DLT calibration method is:

[0045]

[0046] Step S1.2: Arrange and eliminate z c , we get two about m ij The linear equation is:

[0047]

[0048] Step S1.3: Write equation (3-7) in matrix form:

[0049] Lm=0 (3-8)

[0050] In the above formula, L is a matrix with 2n rows and 12 columns, and n is the number of calibration points involved in the calculation;

[0051] Step S1.4: From (3-8), we can see that if m ij Assuming the pixel coordinates and world coordinates of the calibration points as unknowns, the known parameters are the parameters. Then, there are 12 unknowns in equation (3-8). From equation (3-7), it can be seen that one calibration point can correspond to two linear equations. Therefore, if we want to solve the 12 unknowns in equation (3-8), we need n calibration points (n≥6) to obtain the overdetermined equations about the 12 unknowns. Then, we can use the linear least squares method to solve equation (3-8) and solve the transformation coefficient m. ij .

[0052] In step S2, stereo correction is performed according to the following steps:

[0053] Step S2.1: Ray o can be obtained through camera I and camera II respectively a1 p and o a1 p's spatial information, and obtains ray o a1 p and o a1 When solving the coordinates of point P, find the relationship between the world coordinates of the spatial point P and the image coordinates of the corresponding projection points p1 and p2:

[0054]

[0055] Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of the projection points p1 and p2 respectively, (x w ,y w ,z w ,1) is the world homogeneous coordinate of point P; M1 and M2 are the projection matrices of the two cameras; f1 and f2 are the focal lengths of camera I and camera II respectively;

[0056] Step S2.2: Eliminate f1 and f2 to obtain w ,y w ,z w The linear equations

[0057]

[0058] Preferably, in step S3, stereo matching is completed according to the following steps:

[0059] Step S3.1: The spatial three-dimensional coordinates of the spatial point P can be obtained by combining the above equations. Considering the interference of data noise, it is necessary to solve the equations according to the least squares method and rewrite the above equation in the form of a matrix:

[0060]

[0061] The above formula can be simplified to KX = U, where X is the world coordinate of the desired spatial point, and K and U are known vectors that can be obtained by camera calibration. The least squares method can be used to solve it:

[0062]

[0063] In step S4, the observed three-dimensional structure information is restored according to the following steps:

[0064] According to the above formula, the spatial three-dimensional coordinates of each point on the photo are obtained from the photos taken by the binocular vision camera and the internal and external parameters of the camera. All points on the object under test can find corresponding matching points on the image section to restore the three-dimensional geometric shape of the object under test.

[0065] It is easy to understand that those skilled in the art can combine, split, reorganize, etc. the embodiments of the present application to obtain other embodiments based on one or several embodiments provided in the present application, and these embodiments do not exceed the protection scope of the present application.

[0066] The present invention and its implementation methods are described schematically above, and the description is not restrictive. The embodiments shown in the embodiments are only part of the implementation methods of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by the embodiments and designs a structure and an implementation method similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the protection scope of the present invention.

Claims

1. A mobile crane tower docking monitoring system based on machine vision technology, characterized by: Step S1, calibrate the installation position of the binocular camera to obtain the internal and external parameters of the camera for subsequent image processing; Step S2: stereo correction to eliminate image distortion caused by the position difference between the two cameras; Step S3: stereo matching, which is a process for determining the correspondence between images of the same scene captured by two cameras from different perspectives; Step S4: According to the above correspondence, restore the three-dimensional structure information of the scene, that is, the depth map.

2. The mobile crane tower assembly and docking monitoring system based on machine vision technology according to claim 1 is characterized by: In step S1, the camera calibration is completed according to the following steps: Step S1.1: Calibrate the camera without considering the radial distortion, tangential distortion and thin prism distortion of the camera: DLT calibration method is used. The mathematical model of DLT calibration method is: Step S1.2: Arrange and eliminate z c , we get two about m ij The linear equation is: Step S1.3: Write equation (3-7) in matrix form: Lm=0 (3-8) In the above formula, L is a matrix with 2n rows and 12 columns, and n is the number of calibration points involved in the calculation; Step S1.4: From (3-8), we can see that if m ij Assuming the pixel coordinates and world coordinates of the calibration points as unknowns, the known parameters are the parameters. Then, there are 12 unknowns in equation (3-8). From equation (3-7), it can be seen that one calibration point can correspond to two linear equations. Therefore, if we want to solve the 12 unknowns in equation (3-8), we need n calibration points (n≥6) to obtain the overdetermined equations about the 12 unknowns. Then, we can use the linear least squares method to solve equation (3-8) and solve the transformation coefficient m. ij .

3. The mobile crane tower assembly and docking monitoring system based on machine vision technology according to claim 1 is characterized by: In step S2, stereo correction is performed according to the following steps: Step S2.1: Ray o can be obtained through camera I and camera II respectively a1 p and o a1 p's spatial information, and obtains ray o a1 p and o a1 When solving the coordinates of point P, find the relationship between the world coordinates of the spatial point P and the image coordinates of the corresponding projection points p1 and p2: Where (u1, v1, 1) and (u2, v2, 1) are the image homogeneous coordinates of the projection points p1 and p2 respectively, (x w ,y w ,z w ,1) is the world homogeneous coordinate of point P; M1 and M2 are the projection matrices of the two cameras; f1 and f2 are the focal lengths of camera I and camera II respectively; Step S2.2: Eliminate f1 and f2 to obtain w ,y w ,z w The linear equations 4. The mobile crane tower assembly and docking monitoring system based on machine vision technology according to claim 1 is characterized by: In step S3, stereo matching is completed according to the following steps: Step S3.1: The spatial three-dimensional coordinates of the spatial point P can be obtained by combining the above equations. Considering the interference of data noise, it is necessary to solve the equations according to the least squares method and rewrite the above equation in the form of a matrix: The above formula can be simplified to KX = U, where X is the world coordinate of the desired spatial point, and K and U are known vectors that can be obtained by camera calibration. The least squares method can be used to solve it:

5. The mobile crane tower assembly and docking monitoring system based on machine vision technology according to claim 1 is characterized by: In step S4, the observed three-dimensional structure information is restored according to the following steps: According to the above formula, the spatial three-dimensional coordinates of each point on the photo are obtained from the photos taken by the binocular vision camera and the internal and external parameters of the camera. All points on the object under test can find corresponding matching points on the image section to restore the three-dimensional geometric shape of the object under test.

Images