A marine column posture detection method based on dynamic binocular vision

Abstract

The application relates to a marine column posture detection method based on dynamic binocular vision, which comprises the following steps: binocular camera internal parameter calibration; first camera coordinate system and two-axis inclination sensor coordinate system calibration; first camera coordinate system and world coordinate system static calibration; binocular camera external parameter calibration; dynamic calibration of the first camera and the second camera coordinate system and the world coordinate system; marine column axis equation extraction of the first camera and the second camera: using the image of the photographed marine column, the straight lines at the two ends of the marine column in each image are extracted, the two straight lines are fitted into a straight line, and the slope and intercept of the camera pixel coordinate system marine column axis are obtained; obtaining of plane equation formed by the marine column axis and the focal points of the first camera and the second camera; calculation of marine column posture information.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically a method for detecting the attitude of marine piles based on dynamic binocular vision. Background Technology

[0002] Offshore wind resources are abundant, with higher and more stable wind speeds than on land. The open waters of the marine environment provide better conditions for the installation of wind turbines, enabling them to utilize stronger and more continuous sea winds for large-scale wind power generation. Marine piles, fixed to the seabed, provide a stable support structure for the safe operation of wind turbines and effectively reduce the impact of mechanical stress and vibration caused by changes in the marine environment. Accurately detecting the attitude of marine piles can identify structural problems and potential structural damage risks, allowing for appropriate repair and reinforcement measures to ensure the safety and reliability of marine structures. Therefore, the attitude detection of marine piles has become a key focus in offshore wind power engineering.

[0003] Currently, the measurement methods for marine pile installation mainly rely on traditional manual installation of tilt sensors, which carries the risk of personnel and equipment falling into the sea. Non-contact measurement methods, such as lidar, are easily affected by weather changes. Due to interference from ocean waves, traditional static binocular vision measurement methods cannot meet the requirements of the measurement conditions.

[0004] In summary, to address the problems of low efficiency, low automation, and high labor intensity of current measurement methods, a dynamic binocular vision-based method for detecting the attitude of marine piles is proposed. Summary of the Invention

[0005] This invention provides a method for detecting the attitude of marine bollards based on dynamic binocular vision. By organically combining traditional binocular vision with a dual-axis tilt sensor, accurate attitude detection of marine bollards is achieved, enabling reliable operation even under the dynamic impact of ocean waves. Compared to traditional binocular vision methods that require the application of feature markers, this technology eliminates the need for special processing of the marine bollards. By selecting feature lines of the marine bollard to extract the axis, the angle between the marine bollard and the standard vertical axis is obtained, thus achieving attitude detection of the marine bollard. This method improves the safety of personnel while ensuring measurement accuracy. The technical solution is as follows:

[0006] A method for detecting the attitude of marine bollards based on dynamic binocular vision, the technical solution of which includes the following steps:

[0007] Step (1), binocular camera intrinsic parameter calibration: In the calibration field, Zhang Zhengyou's checkerboard calibration method is used to obtain the intrinsic parameters and distortion coefficients of the first and second cameras;

[0008] Step (2), calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system: In the calibration field, fix the positions of the dual-axis tilt sensor and the first camera; use a total station and the first camera to extract the three-dimensional information and pixel information of the target in the common field of view, perform multiple PnP calibrations, and record the angle output of the dual-axis tilt sensor in each calibration process; let the X-axis and Y-axis output angles of the dual-axis tilt sensor be α. i β i There exists a standard horizontal plane X. L OY L Let the standard horizontal plane X be... L OY L Around X L Axis rotation ω i Then around Y L Axis rotation The angle is obtained as a plane parallel to the current biaxial tilt coordinate system, and the angle around Y is... L Axis rotation Angle and the X-axis output angle α of the dual-axis tilt sensor i Equal to each other, we obtain the following angle transformation relationship:

[0009]

[0010] Based on the aforementioned angle transformation relationship, solve for the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system. sc ;

[0011] Step (3), static calibration of the first camera coordinate system and the world coordinate system: make the hull stationary relative to the standard ground, stably install the first camera and the dual-axis tilt sensor, whose positional relationship has been calibrated, onto the hull, and install the second camera in a suitable position; establish the world coordinate system OX. W Y W Z W OZ in the world coordinate system W The axis is always perpendicular to the standard horizontal plane and points vertically upwards, OX W The projection of the line connecting the axis and the centers of the two cameras onto a standard horizontal plane is parallel, OZ W Axis vectors and OX W The direction of the cross product of the axial vectors is OY. W Axial direction;

[0012] Using a total station and a first camera, the 3D points and pixels of the target in the common field of view are extracted respectively. Using PnP calibration, the initial rotation and translation matrices between the world coordinate system and the first camera coordinate system are calculated, denoted as R. wl_0 T wl_0And record the angle outputs α0 and β0 of the dual-axis tilt sensor along the X and Y axes at this time, as the initial angle outputs of the dual-axis tilt sensor; according to the angle conversion relationship in step (2), obtain ω0,

[0013] Step (4), binocular camera extrinsic calibration: keep the ship stationary, take multiple target images in the common field of view of the binocular cameras, and solve for the rotation matrix R and translation matrix T from the first camera coordinate system to the second camera coordinate system.

[0014] Step (5), dynamic calibration of the coordinate systems of the first and second cameras with the world coordinate system: Adjust the position of the ship platform and bring it close to the marine piles; for each binocular camera shot, simultaneously record the angle output of the dual-axis tilt sensor, let i be the i-th shot of the binocular camera, and let α be the X-axis and Y-axis angle output of the dual-axis tilt sensor. i β i Since i > 0, ω can be obtained from the angle transformation relationship in step (2). i , i > 0; ω0 is obtained by combining the initial angle output obtained in step (3). Obtain the rotation matrix R from the world coordinate system to the first camera coordinate system. wl_i Translation matrix T wl_i If i > 0; then, using the rotation matrix R and translation matrix T obtained in step (4) from the first camera coordinate system to the second camera coordinate system, the rotation matrix R from the world coordinate system to the second camera coordinate system can be obtained. wr_i Translation matrix T wr_i , i > 0;

[0015] Step (6), extraction of the axis equation of the marine pile captured by the first and second cameras: using the images of the marine pile captured by the first and second cameras in step (5), extract the straight lines at both ends of the marine pile in each image, fit the two straight lines into one straight line, and obtain the slope and intercept of the marine pile axis in the camera pixel coordinate system.

[0016] Step (7), obtaining the plane equations formed by the ocean column axis with the focal points of the first and second cameras respectively: using the slope and intercept of the ocean column axis in the pixel coordinate system of each camera obtained in step (6), combined with the camera intrinsic parameters in step (1) and the rotation and translation matrices from the world coordinate system to the coordinate systems of the first and second cameras in step (5), the plane equations formed by the ocean column axis and each camera focal point in the world coordinate system are obtained;

[0017] Step (8), calculation of the attitude information of the marine bollard: Based on the axis equation of the marine bollard obtained in step (7), the axis of the marine bollard is cross-multiplied with the normal vector of the plane formed by the focal points of the first camera and the second camera respectively to obtain the vector of the marine bollard axis l in the world coordinate system; the attitude of the axis in the world coordinate system is obtained from the axis vector l.

[0018] Furthermore, in step (2), the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system is solved. sc The method is as follows:

[0019] Let the X-axis and Y-axis output angles of the dual-axis tilt sensor be α. i β i There exists a standard horizontal plane X. L OY L Let the standard horizontal plane X be... L OY L Around X L Axis rotation ω i Then around Y L Axis rotation The angle is obtained as a plane parallel to the current biaxial tilt coordinate system, and the angle around Y is... L Axis rotation Angle and the X-axis output angle α of the dual-axis tilt sensor i Equal to each other, we obtain the following angle transformation relationship:

[0020]

[0021] Let R be the rotation matrix from the total station coordinate system to the first camera coordinate system. qc_i The rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is R. sc ;

[0022] Let R qc_i inverse matrix Then we can obtain R. sc Formula for elements:

[0023]

[0024] Using a target, the total station and the first camera are calibrated using a point-to-point (PnP) method to obtain the rotation matrix R from the total station coordinate system to the first camera coordinate system. qc_i And record the angle output α of the dual-axis tilt sensor each time. i β i Then, the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system is solved. sc .

[0025] Furthermore, in step (5), let the three-dimensional point in the sensor coordinate system be (x...si ,y si ,z si ), i = 0, 1, 2, ..., where i = 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the static calibration condition of step (3); i > 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the dynamic calibration condition of step (5), where the coordinates of the three-dimensional point in the sensor coordinate system are the coordinates of the three-dimensional point in the sensor coordinate system at the i-th shooting time of the binocular camera; thus, the static three-dimensional point (x s0 ,y s0 ,z s0 ) to dynamic 3D point (x si ,y si ,z si The conversion process of );

[0026]

[0027] Based on the rotation matrix relationship:

[0028]

[0029] The rotation matrix from the world coordinate system to the first camera coordinate system is obtained as follows:

[0030]

[0031] Combining step (4), we can see that the rotation matrix R from the world coordinate system to the second camera coordinate system during the ship's dynamic process is solved. wr_i =RR wl_i Translation matrix T wr_i =T+RT wl_i .

[0032] Furthermore, the method for step (6) is as follows:

[0033] Let the equation of the linear pixel coordinate system at both ends of a marine column in a captured camera image be:

[0034]

[0035] Let the slopes of the two lines be k1 and k2 and the intercepts be b1 and b2, respectively. Let the slope and intercept of the fitted line be k and b. Since there are two angle bisectors of the two intersecting lines, the process of extracting the axis of the marine pile is divided into the following three cases.

[0036] Case 1: k1 = k2, the two lines do not intersect, then k = k1 = k2.

[0037] Case 2: k1 ≠ k2 and k1*k2 > 0, the two lines intersect at point (k1 * k2). It can be known

[0038] Case 3: k1≠k2 and k1*k2≤0, the two lines intersect at point (k1≠k2). It can be known

[0039] Through the above three cases, the fitted equation of the marine plume axis v = ku + b can be obtained for each image; the slope k of the marine plume axis in the first camera pixel coordinate system is recorded. L With intercept b L The slope k in the second camera pixel coordinate system R With intercept b R .

[0040] Furthermore, the method for step (7) is as follows:

[0041] Let the world coordinates of a point on the axis of a marine pile be (X... w ,Y w Z w The image at the pixel coordinates of the first camera is (u1, v1), and the intrinsic parameter matrix of the first camera is M. L The following relationship can be obtained:

[0042]

[0043] Further conversion reveals that:

[0044]

[0045] The following transformation relationship can be obtained:

[0046]

[0047] Assume there are pixels (u) on the axis of the marine column measured by the first camera coordinate system. a ,v a ), (u b ,v b ), whose corresponding world coordinate systems are (X a ,Y a Z a ), (X b ,Y b Z b From this, we can know that:

[0048]

[0049] The slope can be obtained from the pixel coordinate system of the first camera. The axis of the marine column in the first camera pixel coordinate system is v = k L u+b L , can be obtained

[0050]

[0051] Further transformation yields

[0052]

[0053] It can be seen that the plane equation in the world coordinate system where the axis of the marine pile is determined by the first camera is:

[0054]

[0055] Similarly, the intrinsic parameter matrix of the second camera is M. R It can be known that

[0056]

[0057] The axis of the marine column in its second camera pixel coordinate system is v = k R u+b R It can be seen that the plane equation of the axis of the marine pile in the second camera in the world coordinate system is:

[0058]

[0059] In summary, the general equation of the straight line containing the axis of the marine pile in the world coordinate system is:

[0060]

[0061] Furthermore, the calculation method for the attitude information of the marine pile in step (8) is as follows:

[0062] Based on the equation of the marine stake axis obtained in step (7), the cross product of the marine stake axis axis and the normal vector of the plane formed by the focal points of the first and second cameras is performed to obtain the axial vector of the marine stake axis axis in the world coordinate system:

[0063]

[0064] The orientation of the axis in the world coordinate system is obtained from the axis vector l: Let the axis vector l be... Marine column axis and OZ in world coordinate system w The angle between the axes is denoted as the axial angle, due to the relationship between the marine pile and the vertical axis OZ in the world coordinate system. w The angle θ between them is ≥0°.

[0065] This invention enables the detection of attitude information for large marine piles. Compared to traditional pile attitude detection methods, it features high automation, reduced manual labor intensity, and long-distance non-contact measurement. Compared to traditional static binocular vision detection methods, it allows for dynamic binocular measurement and the acquisition of marine pile attitude information without the need for attaching feature points. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the process for detecting the attitude of marine piles based on binocular vision, as implemented in this invention.

[0067] Figure 2 This is a schematic diagram of the dual-axis tilt sensor and camera fixing device of the present invention;

[0068] Figure 3 This is a schematic diagram showing the relationship between the dual-axis tilt sensor of the present invention and the standard rotation angle;

[0069] Figure 4 This is a schematic diagram of the dual-axis tilt sensor and camera calibration device of the present invention;

[0070] Figure 5 This is a schematic diagram of camera extrinsic parameter calibration according to an embodiment of the present invention;

[0071] Figure 6 This is a schematic diagram of the spatial distribution of the marine column attitude visual detection device according to the present invention.

[0072] Figure 7 This is a schematic diagram of the imaging of a marine column in the camera pixel coordinate system according to the present invention.

[0073] Figure 8 This is a schematic diagram of the imaging process of the first and second cameras for the marine pile axis in accordance with the present invention.

[0074] Explanation of reference numerals in the attached diagram: 1-Camera; 2-Dual-axis tilt sensor and camera mounting device; 3-Dual-axis tilt sensor; 4-Total station; 5-Checkerboard calibration plate; 6-First camera; 7-Camera lens; 8-Dual-axis tilt sensor; 9-Camera mounting platform; 10-Dual-axis tilt sensor transmission line; 11-First camera transmission line; 12-Computer host; 13-Display screen; 14-Total station; 15-Total station bracket; 16-Second camera; 17-Second camera transmission line interface; 18-Target; 19-Marine column; 20-Marine column edge line; 21-Marine column axis; 22-First camera focus; 23-First camera imaging along the marine column axis; 24-Second camera focus; 25-Second camera imaging along the marine column axis; Detailed Implementation

[0075] like Figure 1As shown in the figure, this invention discloses a method for detecting the attitude of marine columnar structures based on dynamic binocular vision. The invention is further described in conjunction with the accompanying drawings. Figure 1 This includes the following steps:

[0076] Step (1), binocular camera intrinsic parameter calibration: In the calibration field, the binocular camera takes a series of pictures of the checkerboard calibration board. By capturing images of the known calibration board at multiple angles and positions, and using the corresponding Zhang Zhengyou checkerboard calibration algorithm, the camera's intrinsic parameter matrix M and distortion coefficient D are calculated.

[0077]

[0078] D = [d1, d2, p1, p2, d3]

[0079] in, The pixel size is in the horizontal direction. denoted as the vertical pixel size, u0 as the horizontal principal point coordinates, v0 as the vertical principal point coordinates, f as the camera focal length, d1, d2, and d3 as radial distortion coefficients, and p1 and p2 as tangential distortion coefficients.

[0080] In this implementation scheme, each camera captures a total of 20 images of a 9*11 checkerboard pattern with a black and white grid width of 30cm. The resulting intrinsic parameter matrices for the first and second cameras are as follows:

[0081]

[0082]

[0083] The distortion error of the first and second camera lenses is

[0084] D L =[-0.07794494245021506,0.0802940909720152,-0.0008260688651301911,-0.0001261240070349837,0.4251335594716272]

[0085] D R =[-0.08499056488213746,0.2384806547288482,-0.0005699086674039354,0.0001310003886636932,-0.5062749242126027]

[0086] Step (2), calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system: See Figure 2The device combines a first camera and a dual-axis tilt sensor in a fixed relative position. In their 2015 article, "A Relative Pose Measurement Method Combining Vision and Tilt Sensors," Chen Yang et al. mentioned a method for converting the angular output of the dual-axis tilt sensor into a specific rotation angle. For example... Figure 3 As shown, let the X-axis and Y-axis output angles of the tilt sensor be α. i β i There exists a standard horizontal plane X. L OY L Let the standard horizontal plane X be... L OY L Around X L Axis rotation ω i Then around Y L Axis rotation An angle can be used to obtain a plane parallel to the current biaxial tilt coordinate system. Let the angle be around Y. L Axis rotation Angle and the X-axis output angle α of the dual-axis tilt sensor i Equal to each other, we can obtain the following angle transformation relationship.

[0087] The following angle transformation relationship can be obtained.

[0088]

[0089] The article "A Method for Relative Pose Measurement Using a Combination of Vision and Tilt Sensors" mentions a method for calibrating the rotation matrix between a dual-axis tilt coordinate system and a camera coordinate system. Let R be the rotation matrix from the total station coordinate system to the first camera coordinate system. qc_i The rotation matrix from the dual-axis tilt coordinate system to the first camera coordinate system is R. sc .

[0090] Let R qc_i inverse matrix Then we can obtain R. sc Element Formula

[0091]

[0092] Multiple targets are placed within the common field of view of the first camera and the total station. The first camera extracts the pixel information of the targets, and the total station extracts the 3D information of each target in the world coordinate system. The PnP calibration algorithm is used to calculate the rotation matrix R from the total station coordinate system to the first camera coordinate system. qc_i And record the angle output α of the dual-axis tilt sensor each time. i β i The rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system can be solved. sc .

[0093] In this implementation plan, the device is described in reference. Figure 4 The rotation matrix from the total station coordinate system to the first camera coordinate system and the output angle of the dual-axis tilt sensor were obtained using the PnP calibration method. Three measurements were performed in this experiment, and the rotation matrix and the output angle of the dual-axis tilt sensor were obtained as follows:

[0094]

[0095]

[0096]

[0097]

[0098] Therefore, the rotation matrix from the dual-axis tilt coordinate system to the first camera coordinate system is:

[0099]

[0100] Step (3), static calibration of the first camera coordinate system and the world coordinate system: See Figure 5 The ship is brought close to the shore, maintaining a relatively stationary state with respect to the standard horizontal plane. The first camera, which has been calibrated in step (2), along with the dual-axis tilt sensor and fixing device, is then stably mounted onto the ship. The second camera is then stably mounted in an appropriate position. A world coordinate system OX is established. W Y W Z W OZ in the world coordinate system W The axis is always perpendicular to the standard horizontal plane and points vertically upwards, OX W The projection of the line connecting the axis and the centers of the two cameras onto a standard horizontal plane is parallel, OZ W Axis vectors and OX W The direction of the cross product of the axial vectors is OY. W Axial direction.

[0101] Using the first camera, total station, and target, PnP calibration was performed to obtain the rotation and translation matrices from the world coordinate system to the first camera coordinate system, denoted as R. wl_0 T wl_0 And record the angle outputs α0 and β0 of the dual-axis tilt sensor along the X and Y axes at this time, as the initial angle outputs of the dual-axis tilt sensor; according to the angle conversion relationship in step (2), obtain ω0,

[0102] In this implementation scheme, the rotation matrix and translation matrix between the world coordinate system and the first camera coordinate system are:

[0103]

[0104]

[0105] Simultaneously record the output angles of the X and Y axes of the dual-axis tilt sensor at this moment.

[0106]

[0107] Step (4), binocular camera extrinsic calibration: See Figure 5 When the ship platform is stationary, the first and second cameras simultaneously capture images of the target on the shore, extract the target pixel information, and obtain the rotation matrix R and translation matrix T from the first camera coordinate system to the second camera coordinate system through precise calculation and calibration.

[0108] In this implementation plan, the binocular cameras simultaneously capture a total of 40 images of the calibration target within their common field of view. The rotation and translation matrices from the first camera coordinate system to the second camera coordinate system are obtained through binocular calibration.

[0109]

[0110]

[0111] Step (5), dynamic calibration of the first and second camera coordinate systems with the world coordinate system: See [link / reference] Figure 6 The position of the hull platform is adjusted to be close to the marine piles. Due to the impact of waves, the hull is in real-time dynamic. For each binocular camera shot, the angle output of the dual-axis tilt sensor is simultaneously recorded. Let i be the i-th shot from the binocular camera, and let α be the X-axis and Y-axis angle outputs of the dual-axis tilt sensor. i β i Since i > 0, ω can be obtained from the angle transformation relationship in step (2). i , The initial angle output obtained in step (3) is used to calculate ω0. Obtain the rotation matrix R from the world coordinate system to the first camera coordinate system. wl_i Translation matrix T wl_i If i > 0; then, using the rotation matrix R and translation matrix T obtained in step (4) from the first camera coordinate system to the second camera coordinate system, the rotation matrix R from the world coordinate system to the second camera coordinate system can be obtained. wr_i Translation matrix T wr_i , i > 0.

[0112] Let the three-dimensional point in the sensor coordinate system be (x si ,y si ,z si), i = 0, 1, 2, ..., where i = 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the static calibration condition of step (3), and i > 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the dynamic calibration condition of step (5), which is the coordinates of a three-dimensional point in the sensor coordinate system at the i-th shooting time of the binocular camera. Let the three-dimensional point in the world coordinate system be (X w ,Y w Z w In step (3), when the hull is stationary relative to the horizontal plane, ω0 is obtained from the angle output values ​​of the X-axis and Y-axis of the initial dual-axis tilt sensor. but

[0113]

[0114] If the hull platform is close to the marine piles, then

[0115]

[0116] The transformation relationship between the coordinate systems of the dual-axis tilt sensor in the two states from when the hull is stationary relative to the horizontal plane to when the hull platform approaches the marine piles.

[0117]

[0118] After the hull platform approaches the marine pile, the rotation process from the sensor coordinate system to the world coordinate system is as follows:

[0119]

[0120] Thus, the rotation matrix from the world coordinate system to the first camera coordinate system is obtained as follows:

[0121]

[0122] Combining step (4), we can see that, based on the above information, the rotation matrix R from the world coordinate system to the second camera coordinate system during the ship's dynamic process can be solved. wr_i =RR wl_i Translation matrix T wr_i =T+RT wl_i .

[0123] In this implementation plan, the angle of the dual-axis tilt sensor is used. For example.

[0124] At this point, the rotation and translation matrices from the world coordinate system to the first camera coordinate system are:

[0125]

[0126] Rotation and translation matrices from the world coordinate system to the second camera coordinate system:

[0127]

[0128] Step (6), extraction of the axis equation of the marine pile captured by the first and second cameras: Images of the marine pile captured by the first and second cameras in step (5) are used. Canny edge detection is performed on the marine pile images captured by the first and second cameras, and Hough line detection is used to detect straight lines in the images. See [reference] Figure 7 By selecting the edge lines at both ends of the marine column in the image, fitting the two lines into a single line, the slope and intercept of the marine column axis in the camera pixel coordinate system are obtained.

[0129] Let the equation of the linear pixel coordinate system at both ends of a marine column in a captured camera image be:

[0130]

[0131] Let the slopes of the two straight lines be k1 and k2, and the intercepts be b1 and b2, respectively. Let the slope and intercept of the fitted line be k and b. Since there are two angle bisectors of the two intersecting straight lines, the process of extracting the axis of the marine pile can be divided into the following three cases.

[0132] Case 1: k1 = k2, the two lines do not intersect, then k = k1 = k2.

[0133] Case 2: k1 ≠ k2 and k1*k2 > 0, the two lines intersect at point (k1 * k2). It can be known

[0134] Case 3: k1≠k2 and k1*k2≤0, the two lines intersect at point (k1≠k2). It can be known

[0135] Through the above three cases, the fitted equation of the marine plume axis, v = ku + b, can be obtained for each image. The slope k of the marine plume axis in the first camera pixel coordinate system is recorded. L With intercept b L The slope k in the second camera pixel coordinate system R With intercept b R .

[0136] In this implementation plan, the slope and intercept of the marine column axis in the first camera pixel coordinate system and the second camera pixel coordinate system are:

[0137]

[0138] Step (7), obtaining the equation of the plane formed by the axis of the marine pile with the focal points of the first and second cameras respectively: See [link / reference] Figure 8 Using the slope and intercept of the ocean column axis in each camera pixel coordinate system obtained in step (6), combined with the camera intrinsic parameters in step (1) and the rotation and translation matrices from the world coordinate system to the first and second camera coordinate systems in step (5), the equation of the plane formed by the ocean column axis and each camera focus in the world coordinate system is obtained.

[0139] Let the world coordinates of a point on the axis of a marine pile be (X... w ,Y w Z w The image at the pixel coordinates of the first camera is (u1, v1), and the intrinsic parameter matrix of the first camera is M. L The scale factor s1 of the first camera can be obtained from the following relationship:

[0140]

[0141] Further conversion reveals that:

[0142]

[0143] The following transformation relationship can be obtained:

[0144]

[0145] Assume there are pixels (u) on the axis of the marine column measured by the first camera coordinate system. a ,v a ), (u b ,v b ), whose corresponding world coordinate systems are (X a ,Y a Z a ), (X b ,Y b Z b From this, we can know that:

[0146]

[0147] The slope can be obtained from the pixel coordinate system of the first camera. The axis of the marine column in the first camera pixel coordinate system is v = k L u+b L , can be obtained

[0148]

[0149] Further transformation yields

[0150]

[0151] It can be seen that the plane equation in the world coordinate system where the axis of the marine pile is determined by the first camera is:

[0152]

[0153] Similarly, the intrinsic parameter matrix of the second camera is M. R It can be known that

[0154]

[0155] The axis of the marine column in its second camera pixel coordinate system is v = k R u+b R It can be seen that the plane equation of the axis of the marine pile in the second camera in the world coordinate system is:

[0156]

[0157] In summary, the general equation of the straight line containing the axis of the marine pile in the world coordinate system is:

[0158]

[0159] In this implementation plan, the general equation of the straight line containing the axis of the marine pile in the world coordinate system is:

[0160]

[0161] Step (8), calculation of the attitude information of the marine pile: Based on the axis equation of the marine pile obtained in step (7), the cross product of the focus of the first camera and the second camera with the normal vector of the plane formed by the axis of the marine pile is obtained, and the vector of the axis l of the marine pile in the world coordinate system is obtained.

[0162] The normal vectors of the planes formed by the focal points of the first and second cameras and the axis of the marine pile in step (7) are respectively:

[0163]

[0164]

[0165] It can be seen that the vector of the marine pile axis l in the world coordinate system is...

[0166]

[0167] The orientation of the axis in the world coordinate system can be derived from the axis vector l. The relationship between the axis of the marine column and the OZ coordinate system. w The angle between the axes is denoted as the axial angle, due to the relationship between the marine pile and the vertical axis OZ in the world coordinate system. wThe angle θ between them is ≥0°. Based on the above information, the tilt of the marine piles can be determined, and the posture of the marine piles can be adjusted to meet the standard.

[0168] In this implementation plan, the axis vector of the marine column in the world coordinate system is...

[0169]

[0170] At this point, the axial angle of the marine pile is 4.31594°. In this implementation plan, the axial angle of the pile was tested multiple times at different angles, and the error was controlled within ±0.3°, meeting the accuracy requirements.

Claims

1. A method for detecting the attitude of marine piles based on dynamic binocular vision, the technical solution of which includes the following steps: Step (1), binocular camera intrinsic parameter calibration: In the calibration field, Zhang Zhengyou's checkerboard calibration method is used to obtain the intrinsic parameters and distortion coefficients of the first and second cameras; Step (2), calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system: In the calibration field, fix the positions of the dual-axis tilt sensor and the first camera; use a total station and the first camera to extract the three-dimensional information and pixel information of the target in the common field of view, perform multiple PnP calibrations, and record the angle output of the dual-axis tilt sensor in each calibration process; let the X-axis and Y-axis output angles of the dual-axis tilt sensor be α. i β i There exists a standard horizontal plane X. L OY L Let the standard horizontal plane X be... L OY L Around X L Axis rotation ω i Then around Y L Axis rotation The angle is obtained as a plane parallel to the current biaxial tilt coordinate system, and the angle around Y is... L Axis rotation Angle and the X-axis output angle α of the dual-axis tilt sensor i Equal to each other, we obtain the following angle transformation relationship: Based on the aforementioned angle transformation relationship, solve for the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system. sc ; Step (3), static calibration of the first camera coordinate system and the world coordinate system: make the hull relatively stationary with respect to the standard ground, stably install the first camera and the dual-axis tilt sensor, whose positional relationship has been calibrated, onto the hull, and install the second camera in a suitable position; establish the world coordinate system OX. W Y W Z W OZ in the world coordinate system W The axis is always perpendicular to the standard horizontal plane and points vertically upwards, OX W The projection of the line connecting the axis and the centers of the two cameras onto a standard horizontal plane is parallel, OZ W Axis vectors and OX W The direction of the cross product of the axial vectors is OY. W Axial direction; Using a total station and a first camera, the 3D points and pixels of the target in the common field of view are extracted respectively. Using PnP calibration, the initial rotation and translation matrices between the world coordinate system and the first camera coordinate system are calculated, denoted as R. wl_0 T wl_0 And record the angle outputs α0 and β0 of the dual-axis tilt sensor along the X and Y axes at this time, as the initial angle outputs of the dual-axis tilt sensor; according to the angle conversion relationship in step (2), obtain ω0, Step (4), binocular camera extrinsic calibration: keep the ship stationary, take multiple target images in the common field of view of the binocular cameras, and solve for the rotation matrix R and translation matrix T from the first camera coordinate system to the second camera coordinate system. Step (5), dynamic calibration of the coordinate systems of the first and second cameras with the world coordinate system: Adjust the position of the ship platform and bring it close to the marine piles; for each binocular camera shot, simultaneously record the angle output of the dual-axis tilt sensor, let i be the i-th shot of the binocular camera, and let α be the X-axis and Y-axis angle output of the dual-axis tilt sensor. i β i Since i > 0, ω can be obtained from the angle transformation relationship in step (2). i , i > 0; ω0 is obtained by combining the initial angle output obtained in step (3). Obtain the rotation matrix R from the world coordinate system to the first camera coordinate system. wl_i Translation matrix T wl_i If i > 0; then, using the rotation matrix R and translation matrix T obtained in step (4) from the first camera coordinate system to the second camera coordinate system, the rotation matrix R from the world coordinate system to the second camera coordinate system can be obtained. wr_i Translation matrix T wr_i , i > 0; Step (6), extraction of the axis equation of the marine pile captured by the first and second cameras: using the images of the marine pile captured by the first and second cameras in step (5), extract the straight lines at both ends of the marine pile in each image, fit the two straight lines into one straight line, and obtain the slope and intercept of the marine pile axis in the camera pixel coordinate system. Step (7), obtaining the plane equations formed by the ocean column axis with the focal points of the first and second cameras respectively: using the slope and intercept of the ocean column axis in the pixel coordinate system of each camera obtained in step (6), combined with the camera intrinsic parameters in step (1) and the rotation and translation matrices from the world coordinate system to the coordinate systems of the first and second cameras in step (5), the plane equations formed by the ocean column axis and each camera focal point in the world coordinate system are obtained; Step (8), calculation of the attitude information of the marine bollard: Based on the axis equation of the marine bollard obtained in step (7), the axis of the marine bollard is cross-multiplied with the normal vector of the plane formed by the focal points of the first camera and the second camera respectively to obtain the vector of the marine bollard axis l in the world coordinate system; the attitude of the axis in the world coordinate system is obtained from the axis vector l.

2. The method for detecting the attitude of marine piles according to claim 1, characterized in that, In step (2), the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system is solved. sc The method is as follows: Let the X-axis and Y-axis output angles of the dual-axis tilt sensor be α. i β i There exists a standard horizontal plane X. L OY L Let the standard horizontal plane X be... L OY L Around X L Axis rotation ω i Then around Y L Axis rotation The angle is obtained as a plane parallel to the current biaxial tilt coordinate system, and the angle around Y is... L Axis rotation Angle and the X-axis output angle α of the dual-axis tilt sensor i Equal to each other, we obtain the following angle transformation relationship: Let R be the rotation matrix from the total station coordinate system to the first camera coordinate system. qc_i The rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is R. sc ; Let R qc_i inverse matrix Then we can obtain R. sc Formula for elements: Using a target, the total station and the first camera are calibrated using a point-to-point (PnP) method to obtain the rotation matrix R from the total station coordinate system to the first camera coordinate system. qc_i And record the angle output α of the dual-axis tilt sensor each time. i β i Then, the rotation matrix R from the dual-axis tilt coordinate system to the first camera coordinate system is solved. sc .

3. The method for detecting the attitude of marine piles according to claim 1, characterized in that, In step (5), let the three-dimensional point in the sensor coordinate system be (x... si ,y si ,z si ), i = 0, 1, 2, ..., where i = 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the static calibration condition of step (3); i > 0 represents the coordinates of a three-dimensional point in the sensor coordinate system under the dynamic calibration condition of step (5), where the coordinates of the three-dimensional point in the sensor coordinate system are the coordinates of the three-dimensional point in the sensor coordinate system at the i-th shooting time of the binocular camera; thus, the static three-dimensional point (x s0 ,y s0 ,z s0 ) to dynamic 3D point (x si ,y si ,z si The conversion process of ); Based on the rotation matrix relationship: The rotation matrix from the world coordinate system to the first camera coordinate system is obtained as follows: Combining step (4), we can see that the rotation matrix R from the world coordinate system to the second camera coordinate system during the ship's dynamic process is solved. wr_i =RR wl_i Translation matrix T wr_i =T+RT wl_i .

4. The method for detecting the attitude of marine piles according to claim 1, characterized in that, The method for step (6) is as follows: Let the equation of the linear pixel coordinate system at both ends of a marine column in a captured camera image be: Let the slopes of the two lines be k1 and k2 and the intercepts be b1 and b2, respectively. Let the slope and intercept of the fitted line be k and b. Since there are two angle bisectors of the two intersecting lines, the process of extracting the axis of the marine pile is divided into the following three cases. Case 1: k1 = k2, the two lines do not intersect, then k = k1 = k2. Case 2: k1 ≠ k2 and k1*k2 > 0, the two lines intersect at point (k1 * k2). It can be known Case 3: k1≠k2 and k1*k2≤0, the two lines intersect at point (k1≠k2). It can be known Through the above three cases, the fitted equation of the marine plume axis v = ku + b can be obtained for each image; the slope k of the marine plume axis in the first camera pixel coordinate system is recorded. L With intercept b L The slope k in the second camera pixel coordinate system R With intercept b R .

5. The method for detecting the attitude of marine piles according to claim 1, characterized in that, The method for step (7) is as follows: Let the world coordinates of a point on the axis of a marine pile be (X... w ,Y w Z w The image at the pixel coordinates of the first camera is (u1, v1), and the intrinsic parameter matrix of the first camera is M. L Given that the scale factor of the first camera is s1, the following relationship can be obtained: Further conversion reveals that: The following transformation relationship can be obtained: Assume there are pixels (u) on the axis of the marine column measured by the first camera coordinate system. a ,v a ), (u b ,v b ), whose corresponding world coordinate systems are (X a ,Y a Z a ), (X b ,Y b Z b From this, we can know that: The slope can be obtained from the pixel coordinate system of the first camera. The axis of the marine column in the first camera pixel coordinate system is v = k L u+b L , can be obtained Further transformation yields It can be seen that the plane equation in the world coordinate system where the axis of the marine pile is determined by the first camera is: Similarly, the intrinsic parameter matrix of the second camera is M. R It can be known that The axis of the marine column in its second camera pixel coordinate system is v = k R u+b R It can be seen that the plane equation of the axis of the marine pile in the second camera in the world coordinate system is: In summary, the general equation of the straight line containing the axis of the marine pile in the world coordinate system is:

6. The method for detecting the attitude of marine piles according to claim 1, characterized in that, The calculation method for the attitude information of the marine pile in step (8) is as follows: Based on the equation of the marine stake axis obtained in step (7), the cross product of the marine stake axis axis and the normal vector of the plane formed by the focal points of the first and second cameras is performed to obtain the axial vector of the marine stake axis axis in the world coordinate system: The orientation of the axis in the world coordinate system is obtained from the axis vector l: Let the axis vector l be... Marine column axis and OZ in world coordinate system w The angle between the axes is denoted as the axial angle, due to the relationship between the marine pile and the vertical axis OZ in the world coordinate system. w The angle θ between them is ≥0°.

Images