A monocular vision-based bridge hoisting position real-time measurement method and system

Abstract

The application belongs to the technical field of bridge hoisting, and discloses a bridge hoisting pose real-time measurement method and system based on monocular vision, which comprises the following steps: calibrating the relative position between an optical calibration plate and the upper surface of a prefabricated bridge before hoisting operation; taking multiple calibration images of the upper surface of the prefabricated bridge; obtaining the projection transformation matrix of the calibration plate coordinate system and the image coordinate system of any image; calculating the projection coordinates of the vertexes on the upper surface of the prefabricated bridge in the calibration plate coordinate system; obtaining the shape features of the upper surface of the prefabricated bridge based on the multiple calibration images, and calculating the relative position relationship; measuring during hoisting: obtaining the pose of the prefabricated bridge according to real-time images and the relative position relationship obtained through calibration. The application proposes that the relative position mapping relationship between the optical calibration plate and the prefabricated bridge is accurately established by using multi-view image analysis and geometric constraint before hoisting operation, and then the real-time and accurate measurement of the pose of the prefabricated bridge is realized based on monocular vision.

Description

Technical Field

[0001] The present invention belongs to the technical field related to bridge hoisting, and more specifically, relates to a real-time measurement method and system for bridge hoisting posture based on monocular vision. Background Art

[0002] During the bridge hoisting operation of a bridge erection machine, precise control of the six degrees of freedom (three-axis translation and three-axis rotation) of the prefabricated bridge is a core element to ensure structural stability, construction quality, and personnel safety. For example, longitudinal pitch deviation may cause the bridge body to slip, while lateral yaw deviation can easily cause misalignment of prestressed channel steel components and uneven vertical load distribution on the bridge piers. Currently, bridge erection machines generally use a flexible cable lifting system suspended by front and rear beam trolleys. However, due to the lack of real-time six-degree-of-freedom posture data of the hoisted bridge, it is difficult to implement a closed-loop control strategy with active compensation function. Instead, it can only rely on an open-loop control mode. Its under-actuated characteristics, combined with the multi-degree-of-freedom coupled motion of the hoisted bridge, create a technical challenge for high-precision posture control.

[0003] Several solutions attempt to address these challenges. For example, patent CN118854787A uses an inertial measurement unit (IMU) to measure the three-axis rotational attitude of a bridge. However, the IMU relies on the double integration of acceleration signals to calculate three-axis translation. The resulting cumulative error and drift are particularly prominent in high-precision applications, making it difficult to meet reliability requirements.

[0004] In summary, it is difficult to obtain six-degree-of-freedom real-time pose data during the current bridge lifting process. The reliability and accuracy of existing real-time pose data acquisition methods are low, which greatly limits the further improvement of the automation level of bridge erection machines. Summary of the Invention

[0005] In response to the above defects or improvement needs of the prior art, the present invention provides a real-time measurement method and system for bridge hoisting posture based on monocular vision, which is used to solve the current difficulties in obtaining six-degree-of-freedom real-time posture data during bridge hoisting, and the low reliability and accuracy of existing real-time posture data acquisition methods. Its purpose is to measure the six-degree-of-freedom posture of a prefabricated bridge in real time and accurately during the hoisting operation of a bridge erection machine, wherein the six-degree-of-freedom posture includes the three-axis translation and three-axis rotation angle of the prefabricated bridge in three-dimensional space.

[0006] To achieve the above objectives, according to one aspect of the present invention, a method for real-time measurement of bridge hoisting posture based on monocular vision is provided, comprising:

[0007] S1. Before the hoisting operation, the relative position between the optical calibration plate set on the upper surface of the prefabricated bridge and the upper surface of the prefabricated bridge is calibrated, specifically including:

[0008] S11, taking a plurality of calibration images of the upper surface of the prefabricated bridge, wherein any of the calibration images covers the upper surface of the prefabricated bridge;

[0009] S12, for any of the calibration images, solving the projection transformation matrix between the image coordinate system and the calibration plate coordinate system according to the 2D-3D point correspondence between the image coordinate system coordinates of the feature points on the optical calibration plate and the calibration plate coordinate system coordinates;

[0010] S13, for any of the calibration images, calculating the three-dimensional coordinates of the vertex in the calibration plate coordinate system according to the image coordinate system coordinates of the vertex on the upper surface of the prefabricated bridge and the projection transformation matrix; and obtaining the shape characteristics of the upper surface of the prefabricated bridge based on the three-dimensional coordinates of the vertices corresponding to the multiple calibration images, and then calculating the relative positional relationship between the optical calibration plate and the upper surface of the prefabricated bridge;

[0011] S2, measurement during the lifting process, specifically including:

[0012] S21, using a monocular camera installed on the bridge erection machine to obtain a real-time image of the upper surface of the prefabricated bridge, and obtaining a real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system based on the real-time image;

[0013] S22, based on the relative position relationship between the optical calibration plate and the upper surface of the prefabricated bridge, the real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system, and the relative position relationship between the camera coordinate system obtained by calibration and the bridge-erecting machine coordinate system, the position and posture of the prefabricated bridge in the bridge-erecting machine coordinate system is obtained.

[0014] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, S12 specifically includes:

[0015] S121, for any of the calibration images, identifying the image coordinate system coordinates of the feature points;

[0016] S122, based on the 2D-3D point correspondence between the image coordinate system coordinates of the feature points and the calibration plate coordinate system coordinates, solve the candidate solutions of the projection transformation matrix, and screen the feature points that meet the preset threshold requirements through the reprojection error to form an inner point set corresponding to the candidate solutions;

[0017] S123, repeating S122 multiple times to obtain a maximum inlier point set with the largest number of inlier point sets, wherein the candidate solution corresponding to the maximum inlier point set is the initial model of the projection transformation matrix;

[0018] S124 , performing weighted least square optimization on the initial model of the projection transformation matrix based on the feature points in the maximum inlier set according to the reprojection error, to obtain an optimized projection transformation matrix.

[0019] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, the optimization goal of weighted least squares optimization of the initial model of the projection transformation matrix in S124 is:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] ;

[0025] in, To calibrate the coordinate system of the plate to The projection transformation matrix of the image coordinate system of the calibration image, is the projection transformation matrix The optimal value after weighted least squares optimization; For the The optimization weight of feature points; is the maximum interior point set; For the Reprojection error of feature points; For the The initial model of the projection transformation matrix of the calibration image; For the The two-dimensional homogeneous coordinates of the feature points in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm; is the robust standard deviation estimate of the maximum inlier set reprojection error; is the absolute median difference function; Indicates finding the median of the input data set.

[0026] According to the monocular vision-based real-time measurement method for bridge hoisting posture provided by the present invention, S12 further includes:

[0027] S125, based on the maximum interior point set , using the projection transformation matrix after the primary optimization as the initial value, the projection transformation matrix is ​​optimized twice by the LM algorithm to obtain the projection transformation matrix after the secondary optimization;

[0028] Among them, the LM algorithm iterative optimization objective function is constructed as follows:

[0029] ;

[0030] ;

[0031] ;

[0032] in, is the projection transformation matrix The optimal value after LM optimization; is the Huber loss function; the threshold Set according to the absolute median difference criterion; for An estimate of the standard deviation of the measurement noise; The optimization stage of the LM algorithm j Reprojection error of feature points; For the The two-dimensional homogeneous coordinates of the feature points in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm.

[0033] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, S13 specifically includes:

[0034] S131, for any of the calibration images, using a deep learning-based object detection algorithm to identify the image coordinate system coordinates of the vertices on the upper surface of the prefabricated bridge, and obtaining the calibration plate coordinate system coordinates of the vertices based on the projection transformation matrix corresponding to the calibration image;

[0035] S132, optimizing the coordinates of the vertex in the calibration plate coordinate system according to the reprojection errors of the vertices corresponding to the plurality of calibration images, to obtain the optimal coordinates of the vertex in the calibration plate coordinate system;

[0036] S133, obtaining the optimal shape of the upper surface of the precast bridge in the calibration plate coordinate system according to the optimal coordinate fitting of the vertices in the calibration plate coordinate system, and then obtaining the shape characteristics of the upper surface of the precast bridge;

[0037] S134 , obtaining a relative positional relationship between the optical calibration plate and the upper surface of the prefabricated bridge according to shape features of the upper surface of the prefabricated bridge.

[0038] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, the optimization objective in S132 is:

[0039] ;

[0040] in, Represents the global optimal coordinate vector of the vertex in the calibration plate coordinate system; Represents the two-dimensional homogeneous coordinate vectors of all vertices in the calibration plate coordinate system; Represents the two-dimensional homogeneous coordinate vectors of all vertices in the image coordinate system; Represents the calibration image The corresponding projection transformation matrix; Indicates the number of calibration images.

[0041] According to the monocular vision-based real-time measurement method for bridge hoisting posture provided by the present invention, S2 further includes before S22:

[0042] By calculating the rigid transformation relationship between the IMU1 fixed on the bridge-building machine and the IMU2 fixed inside the monocular camera in real time, the relative position relationship between the camera coordinate system and the bridge-building machine coordinate system is calibrated in real time.

[0043] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, S21 obtains the real-time transformation relationship between the calibration plate coordinate system and the camera coordinate system according to the real-time image, and the method also includes:

[0044] Use the IMU2 measurement data to perform real-time image deblurring, including:

[0045] Synchronously obtain the measurement data of IMU2 and the real-time image captured by the monocular camera;

[0046] In the exposure window of each frame of the real-time image, the angular velocity in the IMU2 measurement data is numerically integrated to calculate the three-axis attitude change of the monocular camera;

[0047] Based on the rigid body kinematics model, the motion trajectory of each pixel point of the real-time image on the plane is calculated; then, a motion blur kernel is constructed according to the motion trajectory, and image restoration and deblurring are achieved through frequency domain deconvolution.

[0048] According to the real-time measurement method for bridge hoisting posture based on monocular vision provided by the present invention, S21 obtains the real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system according to the real-time image, specifically including:

[0049] For the real-time image, identifying the image coordinate system coordinates of the feature points;

[0050] Establish the 2D-3D point correspondence between the coordinates of the feature point image coordinate system and the coordinates of the calibration plate coordinate system;

[0051] According to the intrinsic parameters of the monocular camera obtained in advance and based on the 2D-3D point correspondence, the n-point perspective algorithm is used to iteratively optimize and calculate the rotation vector and translation vector from the calibration plate coordinate system to the camera coordinate system.

[0052] According to another aspect of the present invention, a real-time measurement system for bridge hoisting posture based on monocular vision is provided, the system comprising a memory and a processor, the memory storing a computer program, and the processor executing any one of the above-mentioned real-time measurement methods for bridge hoisting posture based on monocular vision when executing the computer program.

[0053] In general, compared with the prior art, the above technical solutions conceived by the present invention provide a method and system for real-time measurement of bridge hoisting posture based on monocular vision:

[0054] 1. This paper proposes using multi-view image analysis to capture the shape features of the precast bridge's upper surface before the hoisting operation. Then, through geometric constraint calibration, a precise mapping relationship between the optical calibration plate and the precast bridge is established. This systematically eliminates the pose analysis errors caused by the tilted calibration plate in traditional solutions. Furthermore, monocular vision is used to achieve real-time and accurate pose measurement of the precast bridge. This effectively improves the real-time, accuracy, and reliability of the precast bridge's pose measurement during the hoisting operation of the bridge-erecting crane, providing reliable data support for the automated closed-loop control of the bridge-erecting crane.

[0055] 2. When obtaining the projection transformation matrix corresponding to the calibration image, a method is proposed to solve and calculate the initial model of the maximum internal point set and the corresponding projection transformation matrix. Based on this, the projection transformation matrix is ​​then optimized using the weighted least squares method and an optimization algorithm, making the obtained projection transformation matrix more accurate and reliable. When obtaining the shape features of the prefabricated bridge surface, a method is proposed to optimize the vertex projection coordinates based on the projection coordinates of the vertices corresponding to multiple calibration images according to the reprojection error through an optimization algorithm. This is conducive to obtaining more accurate and reliable projection coordinates of the vertices on the prefabricated bridge surface, thereby improving the accuracy of relative position calibration.

[0056] 3. A method was proposed to correct camera installation angle in real time during operation through a dual IMU attitude chain solution. This method systematically eliminates the pose analysis errors caused by camera installation deviation in traditional solutions and ensures stable operation of the measurement system under complex working conditions.

[0057] 4. The present invention is aimed at bridge-building machines. It uses optical calibration plates and monocular vision measurement technology, combined with IMU's image deblurring algorithm, sub-pixel corner point recognition algorithm, n-point perspective pose solution algorithm, etc., to measure the six-degree-of-freedom position of the prefabricated bridge being hoisted during the bridge-building machine's hoisting operation in real time, and the measurement results are accurate and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a schematic diagram of the hoisting of a prefabricated bridge provided by an embodiment of the present invention.

[0059] Figure 2 1 is a schematic diagram of the installation of a monocular camera provided by an embodiment of the present invention.

[0060] Figure 3 Schematic diagram of the arrangement of the optical calibration plate on the upper surface of a prefabricated bridge provided in an embodiment of the present invention.

[0061] Figure 4 Schematic diagram of an optical calibration plate provided in an embodiment of the present invention.

[0062] Figure 5 This is a flow chart of a method for real-time measurement of bridge hoisting posture based on monocular vision according to an embodiment of the present invention.

[0063] Throughout the drawings, the same reference numerals are used to denote the same elements or structures, wherein:

[0064] 1- Main beam of bridge-erecting machine, 2- Rear outrigger of bridge-erecting machine, 3- Middle outrigger of bridge-erecting machine, 4- Front outrigger of bridge-erecting machine, 5- Rear beam trolley, 51- Lifting mechanism, 52- Crossbeam of rear beam trolley, 53- Hook, 6- Front beam trolley, 7- Prefabricated bridge, 8- Monocular camera, 9- Optical calibration plate. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0066] See also Figure 1 、 Figure 2 and Figure 5 This embodiment provides a method for real-time measurement of a bridge hoisting posture based on monocular vision, and the method includes:

[0067] S1. Before the hoisting operation, the relative position between the optical calibration plate set on the upper surface of the prefabricated bridge and the upper surface of the prefabricated bridge is calibrated, specifically including:

[0068] S11, taking a plurality of calibration images of the upper surface of the prefabricated bridge, wherein any of the calibration images covers the upper surface of the prefabricated bridge;

[0069] S12, for any of the calibration images, solving the projection transformation matrix between the image coordinate system and the calibration plate coordinate system according to the 2D-3D point correspondence between the coordinates of the feature point image coordinate system on the optical calibration plate and the coordinates of the calibration plate coordinate system;

[0070] S13, for any of the calibration images, calculating the three-dimensional coordinates of the vertex in the calibration plate coordinate system according to the image coordinate system coordinates of the vertex on the upper surface of the prefabricated bridge and the projection transformation matrix; and obtaining the shape characteristics of the upper surface of the prefabricated bridge based on the three-dimensional coordinates of the vertices corresponding to the multiple calibration images, and then calculating the relative positional relationship between the optical calibration plate and the upper surface of the prefabricated bridge;

[0071] S2, measurement during the lifting process, specifically including:

[0072] S21, using a monocular camera installed on the bridge erection machine to obtain a real-time image of the upper surface of the prefabricated bridge, and obtaining a real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system based on the real-time image;

[0073] S22, based on the relative position relationship between the optical calibration plate and the upper surface of the prefabricated bridge, the real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system, and the relative position relationship between the camera coordinate system obtained by calibration and the bridge-erecting machine coordinate system, the position and posture of the prefabricated bridge in the bridge-erecting machine coordinate system is obtained.

[0074] Furthermore, the S11 captured multiple calibration images of the precast bridge's upper surface. Before the lifting operation, the drone was used to capture D images of the bridge's upper surface. The drone's camera had a resolution of 48MP, and the camera's intrinsic parameters were calibrated using the Zhang Zhengyou calibration method before the operation. The captured images completely covered the bridge's upper surface, ensuring that all key features and boundaries were clearly visible.

[0075] In some specific embodiments, S12 obtaining the projection transformation matrix between the image coordinate system and the calibration plate coordinate system specifically includes:

[0076] S121, for any of the calibration images, identifying the image coordinate system coordinates of the feature points;

[0077] S122, based on the 2D-3D point correspondence between the image coordinate system coordinates of the feature points and the calibration plate coordinate system coordinates, solve the candidate solutions of the projection transformation matrix, and screen the feature points that meet the preset threshold requirements through the reprojection error to form the internal point set corresponding to the candidate solution;

[0078] S123, repeat S122 multiple times to obtain a maximum number of inlier sets, and the candidate solution corresponding to the maximum inlier set is an initial model of the projection transformation matrix;

[0079] S124, based on the feature points in the maximum inlier set, performing weighted least squares optimization on the initial model of the projection transformation matrix according to the re-projection error to obtain a once-optimized projection transformation matrix.

[0080] S125, based on the maximum inlier set , taking the once-optimized projection transformation matrix as an initial value, performing secondary optimization on the projection transformation matrix by Levenberg-Marquardt (LM) algorithm to obtain a twice-optimized projection transformation matrix;

[0081] Specifically, referring to Figure 4 , the optical calibration board can be a ChArUco board, and the feature points correspond to the corner points of the ChArUco board. The image coordinate system coordinates of the identified optical calibration board corner points are as follows:

[0082] For each calibration image in the image set , the image coordinate system coordinates of the feature points such as corner points are accurately identified by sub-pixel level corner point detection based on the gray moment algorithm , where is the number of successfully identified corner points. The iteration convergence condition of the gray moment algorithm is that the coordinate change of all corner points of adjacent two iterations respectively satisfies pixels or the number of iterations is greater than or equal to 50 times, where:

[0083] ;

[0084] , the positive direction of the axis in the image coordinate system is the direction, the positive direction of the axis is the direction; , the gradient of the coordinate value in the , direction is the first derivative; , the second derivative is , , and the sub-pixel offset of the coordinate value in the , direction is

[0085] Then, the projection relationship between the image coordinate system of the calibration image and the calibration board coordinate system is calculated.

[0086] Constructing calibration images The projection relationship equation between the image coordinate system and the calibration plate coordinate system is:

[0087] ;

[0088] ;

[0089] in express The two-dimensional homogeneous coordinates of ; To calibrate the coordinate system of the plate to The projection transformation matrix of the image coordinate system of the calibration image; Representing feature points The two-dimensional homogeneous coordinates in the calibration plate coordinate system are composed of the three-dimensional physical coordinates Obtained through degradation treatment.

[0090] Each feature point Two linear equations can be obtained:

[0091] ;

[0092] The Random Sample Consensus (RANSAC) algorithm is used to select feature points that meet the homography constraints as the inliers to eliminate mismatched point pairs, that is, point pairs whose reprojection error exceeds the threshold or whose geometric constraints are not satisfied, and to select the largest inliers. The details are as follows:

[0093] (1) Random selection in each iteration non-collinear points, Points stacked into Matrix :

[0094] ;

[0095] (2) Decomposition by singular value decomposition (SVD) algorithm .in, is the left singular vector matrix; Singular value matrix; Right singular vector matrix. Take The last column (corresponding to the minimum singular value) is used as the solution vector to reconstruct the projection transformation matrix and obtain the candidate solution of the projection transformation matrix .

[0096] (3) Based on the candidate solution, calculate Reprojection error of points ,like Pixels are marked as inliers, and the inlier set is obtained:

[0097] ;

[0098] in, represents the projection function, ; represents the Euclidean norm.

[0099] (4) Repeat (1) (2) (3) multiple times. When the number of iterations is , or the current optimal inlier ratio meets the convergence condition (formula see below), or when the maximum inlier set does not change after 10 consecutive iterations, the iteration terminates and the solution with the largest number of inlier sets is selected. As the initial model; the optimal interior point ratio formula is as follows:

[0100] ;

[0101] in, is the current maximum interior point ratio, i.e. the optimal interior point ratio; For confidence, it is guaranteed that 99% probability of sampling all internal points; is the preset inlier ratio threshold; is the convergence factor to prevent premature termination.

[0102] In the maximum interior point set The weighted least squares method is used to roughly optimize , by weight distribution, the potential noise influence is suppressed and the estimation accuracy of the homography matrix is ​​preliminarily improved. Specifically, the optimization goal of optimizing the initial model of the projection transformation matrix using the weighted least squares method based on the reprojection error in S124 is:

[0103] ;

[0104] ;

[0105] ;

[0106] ;

[0107] in, is the projection transformation matrix The optimal value after weighted least squares optimization; For the The optimization weight of feature points; is the maximum interior point set; For the Reprojection error of feature points; is the initial model of the projection transformation matrix; Feature Points Two-dimensional homogeneous coordinates in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm; is the robust standard deviation estimate of the maximum inlier set reprojection error; is the absolute median difference function; Indicates finding the median of the input data set.

[0108] Furthermore, the projection transformation matrix is ​​iteratively adjusted in the nonlinear space through the LM algorithm. , achieving sub-pixel precision and minimizing the reprojection error.

[0109] Among them, the LM algorithm iterative optimization objective function is constructed as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] in, is the projection transformation matrix The optimal value after LM optimization; is the Huber loss function, which is used to suppress residual anomalies; the threshold Set according to the absolute median difference criterion; for An estimate of the standard deviation of the measurement noise; The optimization stage of the LM algorithm Reprojection error of feature points; is a feature point Two-dimensional homogeneous coordinates in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm.

[0114] Projection transformation matrix during LM algorithm iteration The update method is:

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] in, for stage estimated value of; for The parameter increment matrix of the stage; for Dynamic damping factor of the stage; for Iterative gain ratio of the stage; is the damping adjustment coefficient; for The residual vector of the stage; for Stage residual vector right The Jacobian matrix of ; for The updated weight matrix of the stage; Indicates converting a vector into a diagonal matrix; for Phase I The weights of the inlier residuals; for k stage Reprojection error of the point.

[0123] Furthermore, calculating the relative relationship between the optical calibration plate and the upper surface of the bridge, i.e., S13, specifically includes:

[0124] S131, for any of the calibration images, using a deep learning-based object detection algorithm to identify the image coordinate system coordinates of the vertices on the upper surface of the prefabricated bridge, and obtaining the calibration plate coordinate system coordinates of the vertices based on the projection transformation matrix corresponding to the calibration image;

[0125] S132, optimizing the coordinates of the vertex in the calibration plate coordinate system according to the reprojection errors of the vertices corresponding to the plurality of calibration images, to obtain the optimal coordinates of the vertex in the calibration plate coordinate system;

[0126] S133, fitting the optimal shape of the precast bridge upper surface in the calibration board coordinate system according to the optimal coordinates of the vertex in the calibration board coordinate system, and further obtaining the shape characteristics of the precast bridge upper surface;

[0127] S134, obtaining the relative position relationship between the optical calibration board and the precast bridge upper surface according to the shape characteristics of the precast bridge upper surface.

[0128] In some embodiments, the global optimal coordinate vector of the vertex in the calibration board coordinate system is iteratively calculated by minimizing the re-projection error using the LM algorithm That is, the optimization objective in S132 is:

[0129] ;

[0130] Wherein, represents the global optimal coordinate vector of the vertex in the calibration board coordinate system; represents the two-dimensional homogeneous coordinate vector of all vertices in the calibration board coordinate system; represents the two-dimensional homogeneous coordinate vector of all vertices in the image coordinate system; represents the number of calibration images d The corresponding projection transformation matrix is the projection transformation matrix obtained after the secondary optimization in the above embodiment; D represents the number of calibration images.

[0131] Specifically, the precast bridge upper surface is rectangular, and a target detection algorithm based on deep learning is used to identify the image coordinate system coordinates of the four vertexes of the bridge upper surface in the image When the number of identified vertexes is not equal to 4, the frame image is automatically discarded.

[0132] Preferably, the target recognition algorithm adopts YOLO v11 network architecture, and the algorithm can be trained in a data set containing 1000 bridge upper surface vertex coordinate labels to obtain.

[0133] Suppose the four vertexes of the bridge upper surface are coplanar with the optical calibration board, then the mapping relationship between the bridge upper surface vertex in the calibration board coordinate system and the image coordinate system is:

[0134] ;

[0135] Wherein, represents the two-dimensional homogeneous coordinate of ; represents the two-dimensional homogeneous coordinate of the bridge upper surface vertex in the calibration board coordinate system, which is obtained by degeneration processing from the three-dimensional physical coordinates ;

[0136] ​ ;

[0137] ;

[0138] The residual vector of the stage as follows:

[0139] ;

[0140] in, for stage estimated value.

[0141] Stage residual vector The Jacobian matrix as follows:

[0142] ;

[0143] The update method of the coordinate vector of the vertex in the calibration plate coordinate system in S132 optimization is: stage The update method is as follows:

[0144] ;

[0145] ;

[0146] ;

[0147] ;

[0148] ;

[0149] ;

[0150] ;

[0151] in, for The parameter increment matrix of the stage; for Dynamic damping factor of the stage; for Iterative gain ratio of the stage; is the damping adjustment coefficient; for The updated weight vector of the stage; for Phase I The weight of the vertex residual; for stage Reprojection error of points; Set according to the absolute median difference criterion; for An estimate of the standard deviation of the measurement noise; for stage i The estimated values ​​of the coordinates of the point calibration coordinate system.

[0152] Quadrilateral vertices obtained by algorithm optimization It may not be possible to form a perfect rectangle, so a fitting method based on principal component analysis is used to obtain the optimal shape of the upper surface of the precast bridge in the calibration plate coordinate system according to the optimal coordinate fitting of the vertices in the calibration plate coordinate system, that is, to fit an optimal rectangle, as follows:

[0153] Get the centroid of the quadrilateral on the upper surface of the precast bridge in the calibration plate coordinate system as follows:

[0154] ;

[0155] For vertices Optimal coordinates in the calibration plate coordinate system , decentralized coordinate values as follows:

[0156] ;

[0157] Then calculate the covariance matrix :

[0158] ;

[0159] Pair covariance matrix Perform eigendecomposition:

[0160] ;

[0161] in, represents a diagonal matrix whose diagonal elements are the eigenvalues and , ; represents the eigenvector matrix; and is the orthonormalized eigenvector, is the long axis direction of the optimal rectangle, is the minor axis direction of the optimal rectangle, and the eigenvector is obtained through the eigenvalue.

[0162] According to the shape characteristics of the upper surface of the precast bridge in the calibration plate coordinate system, i.e., the feature vector, the position information of the upper surface of the precast bridge in the calibration plate coordinate system can be obtained. That is, the transformation relationship between the calibration plate coordinate system and the precast bridge coordinate system is represented by the rotation matrix and translation vectors definition:

[0163] ;

[0164] ;

[0165] ;

[0166] in, represents the rotation matrix from the calibration plate coordinate system to the precast bridge coordinate system; represents the translation vector from the calibration plate coordinate system to the precast bridge coordinate system; Indicates the midpoint of the precast bridge coordinate system coordinates of Indicates the midpoint of the calibration plate coordinate system 's coordinates.

[0167] Furthermore, before S22 obtains the position and posture of the prefabricated bridge in the bridge erection machine coordinate system, S2 also includes:

[0168] By calculating the rigid transformation relationship between the IMU1 fixed on the bridge-building machine and the IMU2 fixed inside the monocular camera in real time, the relative position relationship between the camera coordinate system and the bridge-building machine coordinate system is calibrated in real time.

[0169] like Figure 1 and Figure 2 As shown, when the prefabricated bridge is hoisted and installed, the main beam 1 of the bridge-erecting machine is fixed to the installed bridge body through the rear support legs 2, the middle support legs 3 and the front thrust 4 of the bridge-erecting machine. A rear beam trolley crossbeam 52 and a front beam trolley crossbeam are provided between the two oppositely arranged main beams 1 of the bridge-erecting machine. The two crossbeams are correspondingly provided with a rear beam trolley 5 and a front beam trolley 6. Taking the rear beam trolley 5 as an example, it includes a lifting mechanism 51 provided on the rear beam trolley crossbeam 52. The lifting mechanism 51 is connected to a hook 53. The front and rear hooks 53 lift the prefabricated bridge 7 for hoisting. In this embodiment, a monocular camera 8 and a nine-axis IMU 1 are installed on the bridge-erecting machine. Preferably, the installation position of the monocular camera 8 is in front of the rear beam trolley crossbeam 52 of the bridge-erecting machine.

[0170] The monocular camera contains a nine-axis IMU2, and the resolution of the monocular camera is 48MP. The relative positional relationship between the monocular camera and the nine-axis IMU2 was calibrated. After calibration, the monocular camera's attitude data can be obtained based on the attitude data output by the nine-axis IMU2. When the orientation of the IMU2 coordinate system aligns with the camera's coordinate system, the attitude data output by the nine-axis IMU2 is the monocular camera's attitude data.

[0171] In some embodiments, the image coordinate system origin Located in the upper left corner of the image, The axis runs from left to right along the width of the image. The axis extends from top to bottom along the height direction of the image, and the coordinate vector of any point in the image coordinate system is expressed as Indicates that the unit is pixel.

[0172] Camera coordinate system origin Located at the optical center of the smart camera, Axes parallel to the image coordinate system Axis, the positive direction is consistent with the positive direction of the U axis, extending from left to right. Axes parallel to the image coordinate system Axis, the positive direction is consistent with the positive direction of the V axis, extending from top to bottom. The axis is aligned with the camera optical axis, and the positive direction points from front to back to the image plane. The coordinate vector of any point in the camera coordinate system is expressed as , unit is millimeters.

[0173] The coordinate system of the bridge erection machine is a left-hand coordinate system, and its origin is and the origin of the camera coordinate system coincide, The axis is aligned with the direction of gravity acceleration, with downward being positive. The axis is aligned with the longitudinal direction of the main beam of the bridge crane and points from the front leg to the rear leg. The axis is perpendicular to the horizontal plane Axis, aligned with the crossbeam reference axis of the rear beam trolley of the bridge erection machine, along Relative to the positive direction of the axis The axis is to the left, and the coordinate vector of any point in the bridge crane coordinate system is expressed as , unit is millimeters.

[0174] World coordinate system definition The axis is aligned with the gravity vector, with downward being positive, The axis points to magnetic north, The axis points to the east, ensuring that the three axes are perpendicular to each other. The coordinate vector of any point in the world coordinate system is expressed as , unit is millimeters.

[0175] Nine-axis IMU1 、 、 Axis coordinate system of the bridge crane 、 、 The axis directions are consistent, so the attitude data output by IMU1 represents the attitude data of the bridge-building machine coordinate system, which is convenient for calculation.

[0176] like Figure 3 As shown, an optical calibration plate 9 is posted in the middle position of the upper surface of the prefabricated bridge 7, and the monocular camera can completely capture the optical calibration plate 9.

[0177] Origin of the precast bridge coordinate system Located at the geometric center of the upper surface of the precast bridge, The axis is along the longitudinal reference axis of the precast bridge, and the positive direction extends from the tail to the bow. The axis is along the transverse reference axis of the precast bridge. Axis square perspective The positive direction of the axis extends from the left to the right. The axis is perpendicular to the upper surface of the precast bridge, and the positive direction points to the inside of the structure, that is, perpendicular to the upper surface and downward. The coordinate vector of any point in the precast bridge coordinate system is expressed as , unit is millimeters.

[0178] Calibration plate coordinate system origin Located in the upper left corner of the optical calibration plate, The axis is along the width of the board, with the positive direction extending from left to right. The axis is along the height of the board, with the positive direction extending from top to bottom. The axis is perpendicular to the plane of the optical calibration plate, and the positive direction points to the inner side of the plate, that is, the side where the prefabricated bridge is located. The coordinate vector of any point in the calibration plate coordinate system is expressed as , unit is millimeters.

[0179] Preferably, the ChArUco board is made of an alumina substrate, has anti-deformation properties and diffuse reflection surface treatment, and its physical parameters are configured as follows: checkerboard array size 12×9, single square side length 40mm, ArUco code mark size 30mm.

[0180] Those skilled in the art should understand that the methods for establishing the above-mentioned coordinate systems include but are not limited to the configurations described in the embodiments, and any equivalent scheme for achieving posture measurement through spatial reference system transformation falls within the scope of protection of this patent.

[0181] Furthermore, during the hoisting process, the relative position of the monocular camera and the bridge-building crane is calibrated in real time. By calculating the rigid transformation relationship between the nine-axis IMU1 and the nine-axis IMU2 in real time, the relative position change between the camera coordinate system and the bridge-building crane coordinate system caused by the movement of the bridge-building crane can be dynamically compensated.

[0182] Nine-axis IMU1 measures four elements , the quaternion measured by the nine-axis IMU2 ,but:

[0183] ;

[0184] ;

[0185] in, Represents the rotation matrix from the world coordinate system to the bridge crane coordinate system; Represents the rotation matrix from the world coordinate system to the camera coordinate system.

[0186] Then the rotation matrix from the camera coordinate system to the bridge crane coordinate system is The calculation is as follows:

[0187] .

[0188] Furthermore, during the operation, the optical calibration plate is used to measure the position and posture of the prefabricated bridge in real time. In S21, the real-time transformation relationship between the calibration plate coordinate system and the camera coordinate system is obtained based on the real-time image, which also includes:

[0189] Use the IMU2 measurement data to perform real-time image deblurring, including:

[0190] The IMU2 measurement data and the real-time image captured by the monocular camera are synchronously acquired. The nine-axis IMU2 data and the image exposure time of the monocular camera are strictly synchronized using a hardware trigger signal to establish a unified time and space timestamp reference.

[0191] In the exposure window of each frame of the real-time image, the angular velocity in the IMU2 measurement data is numerically integrated to calculate the three-axis attitude change of the monocular camera;

[0192] Based on the rigid body kinematics model, the motion trajectory of each pixel point of the real-time image on the plane is calculated; then, a motion blur kernel is constructed according to the motion trajectory, and image restoration and deblurring are achieved through frequency domain deconvolution.

[0193] In S21, the real-time transformation relationship between the calibration plate coordinate system and the camera coordinate system is obtained based on the real-time image. The rigid body transformation from the calibration plate coordinate system to the camera coordinate system is iteratively calculated using the perspective-n-point (PnP) algorithm. Specifically, the following steps are performed:

[0194] For the real-time image, identifying the image coordinate system coordinates of the feature points;

[0195] Establish the 2D-3D point correspondence between the coordinates of the feature point image coordinate system and the coordinates of the calibration plate coordinate system;

[0196] According to the intrinsic parameters of the monocular camera obtained in advance and based on the 2D-3D point correspondence, the n-point perspective algorithm is used to iteratively optimize and calculate the rotation vector and translation vector from the calibration plate coordinate system to the camera coordinate system.

[0197] Specifically, the image coordinates of feature points are identified from images captured in real time by a monocular camera.

[0198] Taking the corner point as an example, for the image captured by the monocular camera in real time, the image coordinate system coordinates of the corner point are accurately identified by sub-pixel corner point detection based on the grayscale moment algorithm. ,in is the number of successfully identified corner points. The iterative convergence condition is: the change in the corner point coordinates between two adjacent iterations Pixels or iterations ≥ 50:

[0199] ;

[0200] in, 、 for 、 Directional gradient; 、 is the second-order derivative; 、 This is the sub-pixel offset.

[0201] Establish the 2D-3D point correspondence between the detected corner image coordinates and the calibration plate coordinates. In the case of , based on the 2D-3D point correspondence, the n-point perspective algorithm is used to iteratively optimize the calculation of the rotation vector from the calibration plate coordinate system to the camera coordinate system and translation vectors .

[0202] Then calculate the real-time pose of the prefabricated bridge. That is, calculate the rotation matrix from the prefabricated bridge coordinate system to the bridge erection machine coordinate system and translation vectors as follows:

[0203] ;

[0204] ;

[0205] in, represents the rotation matrix from the precast bridge coordinate system to the calibration plate coordinate system; represents the translation vector from the precast bridge coordinate system to the calibration plate coordinate system; Represents the translation vector from the camera coordinate system to the bridge crane coordinate system.

[0206] Furthermore, this embodiment also provides a real-time measurement system for bridge hoisting posture based on monocular vision, the system including a memory and a processor, the memory storing a computer program, and the processor executing any one of the above-mentioned real-time measurement methods for bridge hoisting posture based on monocular vision when executing the computer program.

[0207] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A real-time measurement method for bridge hoisting posture based on monocular vision, characterized in that: include: S1. Before the hoisting operation, the relative position between the optical calibration plate set on the upper surface of the prefabricated bridge and the upper surface of the prefabricated bridge is calibrated, specifically including: S11, taking a plurality of calibration images of the upper surface of the prefabricated bridge, wherein any of the calibration images covers the upper surface of the prefabricated bridge; S12, for any of the calibration images, solving the projection transformation matrix between the image coordinate system and the calibration plate coordinate system according to the 2D-3D point correspondence between the image coordinate system coordinates of the feature points on the optical calibration plate and the calibration plate coordinate system coordinates; S13, for any of the calibration images, calculating the three-dimensional coordinates of the vertex in the calibration plate coordinate system according to the image coordinate system coordinates of the vertex on the upper surface of the prefabricated bridge and the projection transformation matrix; and obtaining the shape characteristics of the upper surface of the prefabricated bridge based on the three-dimensional coordinates of the vertices corresponding to the multiple calibration images, and then calculating the relative positional relationship between the optical calibration plate and the upper surface of the prefabricated bridge; S2, measurement during the lifting process, specifically including: S21, using a monocular camera installed on the bridge erection machine to obtain a real-time image of the upper surface of the prefabricated bridge, and obtaining a real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system based on the real-time image; S22, based on the relative position relationship between the optical calibration plate and the upper surface of the prefabricated bridge, the real-time transformation relationship from the calibration plate coordinate system to the camera coordinate system, and the relative position relationship between the camera coordinate system obtained by calibration and the bridge-erecting machine coordinate system, the position and posture of the prefabricated bridge in the bridge-erecting machine coordinate system is obtained.

2. The method for real-time measurement of bridge hoisting posture based on monocular vision according to claim 1, characterized in that: S12 specifically includes: S121, for any of the calibration images, identifying the image coordinate system coordinates of the feature points; S122, based on the 2D-3D point correspondence between the coordinates of the feature point image coordinate system and the coordinates of the calibration plate coordinate system, solve the candidate solutions of the projection transformation matrix, and screen the feature points that meet the preset threshold requirements through the reprojection error to form the internal point set corresponding to the candidate solutions; S123, repeating S122 multiple times to obtain a maximum inlier point set with the largest number of inlier point sets, wherein the candidate solution corresponding to the maximum inlier point set is the initial model of the projection transformation matrix; S124 , performing weighted least square optimization on the initial model of the projection transformation matrix based on the feature points in the maximum inlier set according to the reprojection error, to obtain an optimized projection transformation matrix.

3. The real-time measurement method for bridge hoisting posture based on monocular vision according to claim 2 is characterized in that: The optimization goal of weighted least squares optimization of the initial model of the projection transformation matrix in S124 is: ； ； ； ； ； in, To calibrate the coordinate system of the plate to The projection transformation matrix of the image coordinate system of the calibration image, is the projection transformation matrix The optimal value after weighted least squares optimization; For the The optimization weight of feature points; is the maximum interior point set; For the Reprojection error of feature points; For the The initial model of the projection transformation matrix of the calibration image; For the The two-dimensional homogeneous coordinates of the feature points in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm; is the robust standard deviation estimate of the maximum inlier set reprojection error; is the absolute median difference function; Indicates finding the median of the input data set.

4. The method for real-time measurement of bridge hoisting posture based on monocular vision according to claim 2, characterized in that: The S12 also includes: S125, based on the maximum interior point set , using the projection transformation matrix after the primary optimization as the initial value, the projection transformation matrix is ​​optimized twice by the LM algorithm to obtain the projection transformation matrix after the secondary optimization; Among them, the LM algorithm iterative optimization objective function is constructed as follows: ； ； ； in, is the projection transformation matrix The optimal value after LM optimization; is the Huber loss function; the threshold Set according to the absolute median difference criterion; for An estimate of the standard deviation of the measurement noise; The optimization stage of the LM algorithm Reprojection error of feature points; For the The two-dimensional homogeneous coordinates of the feature points in the calibration plate coordinate system; For the The first calibration image The image coordinate system coordinates of the feature points; represents the projection function, ; represents the Euclidean norm.

5. The method for real-time measurement of bridge hoisting posture based on monocular vision according to any one of claims 1 to 4, characterized in that: S13 specifically includes: S131, for any of the calibration images, using a deep learning-based object detection algorithm to identify the image coordinate system coordinates of the vertices on the upper surface of the prefabricated bridge, and obtaining the calibration plate coordinate system coordinates of the vertices based on the projection transformation matrix corresponding to the calibration image; S132, optimizing the coordinates of the vertex in the calibration plate coordinate system according to the reprojection errors of the vertices corresponding to the plurality of calibration images, to obtain the optimal coordinates of the vertex in the calibration plate coordinate system; S133, obtaining the optimal shape of the upper surface of the precast bridge in the calibration plate coordinate system according to the optimal coordinate fitting of the vertices in the calibration plate coordinate system, and then obtaining the shape characteristics of the upper surface of the precast bridge; S134 , obtaining a relative positional relationship between the optical calibration plate and the upper surface of the prefabricated bridge according to shape features of the upper surface of the prefabricated bridge.

6. The method for real-time measurement of bridge hoisting posture based on monocular vision according to claim 5, characterized in that: The optimization goal in S132 is: ； in, Represents the global optimal coordinate vector of the vertex in the calibration plate coordinate system; Represents the two-dimensional homogeneous coordinate vectors of all vertices in the calibration plate coordinate system; Represents the two-dimensional homogeneous coordinate vectors of all vertices in the image coordinate system; Represents the calibration image d The corresponding projection transformation matrix; D Indicates the number of calibration images.

7. The method for real-time measurement of bridge hoisting posture based on monocular vision according to any one of claims 1 to 4, characterized in that: S2 also includes before S22: By calculating the rigid transformation relationship between the IMU1 fixed on the bridge-building machine and the IMU2 fixed inside the monocular camera in real time, the relative position relationship between the camera coordinate system and the bridge-building machine coordinate system is calibrated in real time.

8. The method for real-time measurement of bridge hoisting posture based on monocular vision according to claim 7, characterized in that: In S21, the real-time transformation relationship between the calibration plate coordinate system and the camera coordinate system is obtained based on the real-time image. Previously, it also included: Use the IMU2 measurement data to perform real-time image deblurring, including: Synchronously obtain the measurement data of IMU2 and the real-time image captured by the monocular camera; In the exposure window of each frame of the real-time image, the angular velocity in the IMU2 measurement data is numerically integrated to calculate the three-axis attitude change of the monocular camera; Based on the rigid body kinematics model, the motion trajectory of each pixel point of the real-time image on the plane is calculated; then, a motion blur kernel is constructed according to the motion trajectory, and image restoration and deblurring are achieved through frequency domain deconvolution.

9. The method for real-time measurement of bridge hoisting posture based on monocular vision according to any one of claims 1 to 4, characterized in that: In S21, the real-time transformation relationship between the calibration plate coordinate system and the camera coordinate system is obtained according to the real-time image, which specifically includes: For the real-time image, identifying the image coordinate system coordinates of the feature points; Establish the 2D-3D point correspondence between the coordinates of the feature point image coordinate system and the coordinates of the calibration plate coordinate system; According to the intrinsic parameters of the monocular camera obtained in advance and based on the 2D-3D point correspondence, the n-point perspective algorithm is used to iteratively optimize and calculate the rotation vector and translation vector from the calibration plate coordinate system to the camera coordinate system.

10. A real-time measurement system for bridge hoisting posture based on monocular vision, characterized in that: The system includes a memory and a processor, the memory stores a computer program, and the processor executes the real-time measurement method for bridge hoisting posture based on monocular vision described in any one of claims 1 to 9 when executing the computer program.

Images