A method and system for monitoring the holes of a bridge girder hoist based on monocular machine vision

Abstract

The application relates to a kind of based on monocular machine vision's bridge girder erection machine sling hole monitoring method, comprising: according to control point target establishment target local coordinate system;Control point target includes the reference target installed on the beam top of the beam body to be hoisted with sling hole and shared with sling hole, and a plurality of positioning targets installed on the longitudinal beam of bridge girder erection machine sling, target local coordinate system includes the reference target local coordinate system corresponding to reference target, and the positioning target local coordinate system corresponding to positioning target;The image containing control point target and sling hole is collected by monocular camera installed on bridge girder erection machine sling;According to the pose of positioning target is solved according to image, to obtain the pose degree of freedom of sling.The application converts complex space hole problem into pose solution under two-dimensional coordinate system;According to the pose degree of freedom of sling is solved according to image, the motion state of sling is comprehensively monitored.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bridge intelligent construction and monitoring, and particularly relates to a method and system for monitoring the hole pairing of a bridge girder erection machine hoist based on monocular machine vision. BACKGROUND

[0002] The bridge girder erection machine is a standardized bridge precast component erection device. In terms of operation content, the main function of the bridge girder erection machine is to integrally hoist a precast main girder or hoist a longitudinally segmented segmental girder section by section. The precast main girder and the segmental girder section have holed holes reserved thereon for anchoring of the hoist of the bridge girder erection machine to hoist the girder body.

[0003] In the related art, the main steps of hoisting the girder body by the bridge girder erection machine include hoist hole pairing, hoist anchoring, hoisting, and girder lowering. The existing hoist is mainly a slide rod type jib, and manual adjustment of the jib distance is required to adapt to the holed hole spacing. The hoist hole pairing positioning monitoring is mainly based on manual naked eye observation of the holed hole-hoist anchor rod in the monitoring screen, and the human eye observation has high dependence and poor measurement accuracy. In the related art, some solutions use binocular vision technology to monitor the hoist hole pairing, but the solution has problems such as difficulty in calibration of binocular devices, low long-distance measurement accuracy, and the need for additional sensors for hoist self-positioning, and has high requirements for device calibration and small monitoring field of view. SUMMARY

[0004] The present application provides a method and system for monitoring the hoist hole pairing of a bridge girder erection machine based on monocular machine vision, which solves the technical problems of high dependence on human eye observation, poor measurement accuracy, or high requirements for device calibration and small monitoring field of view in the related art.

[0005] The present application provides a method for monitoring the hoist hole pairing of a bridge girder erection machine based on monocular machine vision, which includes the following steps:

[0006] A target local coordinate system is established according to a control point target. The control point target includes a reference target installed on the top of a hoisted girder body having a holed hole and coplanar with the holed hole, and a plurality of positioning targets installed on the longitudinal beam of the hoist of the bridge girder erection machine. The target local coordinate system includes a reference target local coordinate system corresponding to the reference target and a positioning target local coordinate system corresponding to the positioning target. ;

[0007] An image containing the control point target and the holed hole is collected by a monocular camera installed on the hoist of the bridge girder erection machine.

[0008] The pose of the positioning target is solved according to the image to obtain the pose freedom of the hoist.

[0009] In one embodiment, the step of solving the pose of the positioning target according to the image to obtain the pose freedom of the hoist includes:​

[0010] Based on the image, calculate the local coordinates of the positioning target in the local coordinate system of the reference target. The following positioning coordinates ;

[0011] Calculate the tilt vector of the positioning target. ;

[0012] Calculate the center of the anchor bolt installed on the longitudinal beam of the lifting device in the local coordinate system of the reference target. Plane coordinates below ;

[0013] Repeat the above steps to obtain the pose of each of the positioning targets. and the corresponding anchor bolt plane coordinates , i Indicates the number of the positioning target / anchor;

[0014] Calculate all localization targets The average value of the values ​​is used as the elevation of the lifting device. Vertical tilt angle Lateral tilt angle Turning angle Thus, the orientational degrees of freedom of the lifting device are obtained.

[0015] In one embodiment, the step of calculating the local coordinates of the positioning target in the reference target local coordinate system based on the image is... The following positioning coordinates include:

[0016] Identify the center point of the control point target and the pixel coordinates of the control points in the image; wherein, the control point target has a central circle located at the center, and a plurality of control circles that are symmetrical about the center circle, the center of the central circle is defined as the center point, and the center of the control circles is defined as the control point;

[0017] Based on the pixel coordinates of the center point and control points, the PnP pose estimation algorithm is used to calculate the local coordinate system of the reference target. Transformation matrix to camera coordinate system The local coordinate system of the positioning target Transformation matrix to the camera coordinate system ;

[0018] Calculate the local coordinate system of the positioning target to the local coordinate system of the reference target Translation transformation vector ,by element As the positioning target in the local coordinate system of the reference target a positioning coordinate of the positioning target in the reference target local coordinate system;

[0019] The calculation formula is: .

[0020] In an embodiment, the calculation of the inclination vector of the positioning target includes:

[0021] calculating a rotation transformation matrix of the positioning target local coordinate system to the reference target local coordinate system ;

[0022] The calculation formula is: ;

[0023] According to the rotation transformation matrix , the inclination vector of the positioning target is calculated;

[0024] The calculation formula is: ;

[0025] ;

[0026] wherein, represents the third column vector of the rotation transformation matrix , represents the sign function that outputs 1 when the variable takes a negative number, and outputs 0 when the variable takes other values; represents the first element of the vector .

[0027] In an embodiment, the bridge crane spreader hole monitoring method further includes:

[0028] calculating a plane coordinate of the hoisting hole in the reference target local coordinate system based on the image, as a positioning result of the hoisting hole;

[0029] The calculation of the plane coordinate of the hoisting hole in the reference target local coordinate system based on the image, as a positioning result of the hoisting hole includes:

[0030] calculating a hoisting hole center pixel coordinate in the image based on the image;

[0031] identifying all control point pixel coordinates ,..., , n on the reference target in the image;

[0032] ​based on the coordinates of all control points on the reference target below , construct overdetermined equations ;

[0033] wherein, ;

[0034] ;

[0035] perform singular value decomposition, and take the right singular vector corresponding to the minimum singular value as the solution of ;

[0036] calculate the coordinates of the center of the lifting hole in the reference target local coordinate system , wherein the plane coordinates are taken as the positioning result of the lifting hole;

[0037] The calculation formula is: .

[0038] In an embodiment, the bridge crane lifting tool hole monitoring method further comprises:

[0039] controlling the lifting tool-hole alignment based on the pose degrees of freedom of the lifting tool;

[0040] The lifting tool-hole alignment control based on the pose degrees of freedom of the lifting tool comprises:

[0041] The lifting tool pose controller receives the elevation , vertical inclination , horizontal inclination , rotation angle and the plane coordinates of the anchor rod ;

[0042] The lifting tool pose controller performs PID control through a lifting tool optimization objective function to drive the lifting tool pose actuator to adjust the pose of the lifting tool until the lifting tool optimization objective function converges; the lifting tool optimization objective function includes an elevation optimization objective function and a pose optimization objective function ;

[0043] The elevation optimization objective function is expressed as: ;

[0044] The pose optimization objective function is expressed as: ;

[0045] wherein, represents the target elevation at the preset alignment adjustment time; ​

[0046] The spreader pose controller drives the spreader pose actuator to adjust the planar position of the spreader and its anchor rod until the planar hole-pair optimization objective function converges by PID control of the planar hole-pair optimization objective function.

[0047] The planar hole-pair optimization objective function is expressed as: ;

[0048] wherein, N is the number of positioning targets / anchor rods;

[0049] The spreader pose actuator is driven to make the spreader fall until the anchor rods of the spreader are inserted into the hanger holes.

[0050] The application also provides a single-machine-vision-based spreader hole-pair monitoring system for a bridge girder erection machine, which applies any one of the single-machine-vision-based spreader hole-pair monitoring methods described above and comprises:

[0051] Control point targets for establishing a target local coordinate system, the control point targets comprising a reference target installed on the top of a beam body with hanger holes and coplanar with the hanger holes and a plurality of positioning targets installed on the longitudinal beam of the spreader of the bridge girder erection machine, the target local coordinate system comprising a reference target local coordinate system corresponding to the reference target and a positioning target local coordinate system corresponding to the positioning targets; ;

[0052] A monocular camera installed on the spreader of the bridge girder erection machine and configured to collect an image containing the control point targets and the hanger holes;

[0053] An edge computing terminal configured to obtain the image, solve the pose of the positioning targets according to the image to obtain the pose freedom of the spreader, and output feedback information.

[0054] In an embodiment, the single-machine-vision-based spreader hole-pair monitoring system further comprises:

[0055] A spreader pose controller configured to receive the feedback information output by the edge computing terminal, output a spreader pose adjustment instruction based on the pose freedom of the spreader, and output the spreader pose adjustment instruction;

[0056] A spreader pose actuator configured to receive the spreader pose adjustment instruction output by the spreader pose controller and adjust the pose of the spreader.

[0057] In an embodiment, the single-machine-vision-based spreader hole-pair monitoring system further comprises a light supplementing lamp close to the monocular camera for supplementing light for the hanger holes and the control point targets.

[0058] ​In an embodiment, the spreader pose actuator comprises a hoist trolley, a slewing adjusting mechanism, a lateral adjusting mechanism, and a vertical adjusting mechanism, which respectively receive the spreader pose adjusting instructions output by the spreader pose controller to adjust the spreader height and / or plane position, the slewing angle, the lateral tilt angle, and the vertical tilt angle.

[0059] The technical scheme provided by the embodiments of the present application has the following beneficial effects:

[0060] The method comprises the following steps: first, setting a quick-mountable / replacable reference target and a positioning target on a to-be-lifted beam body and a bridge girder erection machine spreader respectively; the reference target does not need to be pre-calibrated, and control point targets can be arranged flexibly according to different types of spreaders, so as to establish a target local coordinate system with strong adaptability; then, collecting an image containing the control point target and the lifting hole by using a monocular camera, converting a complex space-to-hole problem into a pose solving problem in a two-dimensional coordinate system, and reducing the algorithm complexity; finally, obtaining the pose freedom of the spreader according to the image solving result of the positioning target, avoiding the dependence of the human eye observation scheme, improving the measurement accuracy, and comprehensively monitoring the motion state of the spreader. Only a monocular camera and a control point target are needed in the above process, without the need of high-cost sensors such as a laser radar and a binocular camera, and the requirement for equipment calibration is low,

[0061] Moreover, compared with the binocular vision scheme, the binocular stereo measurement needs an overlap between two camera pictures, and only the overlapping area is the positioning monitoring area, so if the same camera and the same focal length lens are installed at the same installation position, the effective pixels in the monocular camera imaging provided by the embodiments of the present application are more, the scene that can be monitored is larger, the closer position that can be monitored is larger, and the adjustment of the hole-to-hole positioning is more accurate, because the pixel distance of the hole-to-hole positioning in the camera imaging is larger. At the same time, because the effective pixels in the monocular camera imaging are more, the imaging pixel proportion factor (mm / px) of the monitoring area can be made smaller and more accurate, so the measurement accuracy of the embodiments of the present application is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical schemes in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0063] Figure 1 The flow chart of the method for monitoring the hole-to-hole positioning of the bridge girder erection machine spreader based on monocular machine vision in an embodiment of the present application.

[0064] Figure 2A layout schematic diagram of a single-camera-vision-based bridge girder launching machine sling hole monitoring system in an embodiment of the present application.

[0065] Figure 3 A schematic diagram of a control point target in an embodiment of the present application.

[0066] Figure 4 A schematic diagram of a control point target and its target local coordinate system in the field of view of a single camera in an embodiment of the present application.

[0067] In the figure: 1, control point target; 11, reference target; 12, positioning target; 2, beam body to be hoisted; 20, hoisting hole; 3, sling; 31, longitudinal beam; 32, anchor rod; 4, single camera; 5, light supplementing lamp; 6, edge computing terminal; 7, sling pose controller; 8, sling pose actuator; 81, hoist trolley; 82, rotary adjusting mechanism; 83, transverse adjusting mechanism; 84, vertical adjusting mechanism. DETAILED DESCRIPTION

[0068] In order to enable personnel in the art to better understand the present application scheme, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0069] The embodiments of the present application provide a single-camera-vision-based bridge girder launching machine sling hole monitoring method and system, aiming to solve the technical problems of high dependence on human eye observation, poor measurement accuracy, high requirement for device calibration, and small monitoring field of view in the related art.

[0070] As shown in Figure 1 and Figure 2 , wherein, Figure 1 A flowchart of a single-camera-vision-based bridge girder launching machine sling hole monitoring method in an embodiment of the present application. Figure 2 A layout schematic diagram of a single-camera-vision-based bridge girder launching machine sling hole monitoring system in an embodiment of the present application.

[0071] The embodiments of the present application provide a single-camera-vision-based bridge girder launching machine sling hole monitoring method, which comprises the following steps:

[0072] Step S1, establishing a target local coordinate system according to a control point target 1; wherein the control point target 1 comprises a reference target 11 installed on the top of a beam body 2 to be hoisted having a hoisting hole 20 and coplanar with the hoisting hole 20, and a plurality of positioning targets 12 installed on a longitudinal beam of a bridge girder launching machine sling 3, and the target local coordinate system comprises a reference target local coordinate system corresponding to the reference target , and a positioning target local coordinate system corresponding to the positioning target ;

[0073] Step S2, an image containing the control point target 1 and the lifting hole 20 is collected by the monocular camera 4 installed on the bridge girder erection machine lifting appliance 3.

[0074] Step S3, the pose of the positioning target 12 is calculated according to the image, so as to obtain the pose freedom degree of the lifting appliance 3.

[0075] The embodiment provides a monocular machine vision-based bridge girder erection machine lifting appliance hole monitoring method. First, a reference target and a positioning target which can be quickly installed / replaced are respectively arranged on a to-be-lifted beam body and a bridge girder erection machine lifting appliance. The reference target does not need to be pre-calibrated, and the control point target can be flexibly arranged according to different types of lifting appliances, so as to establish a target local coordinate system with strong adaptability. Then, an image containing the control point target and the lifting hole is collected by a monocular camera, a complex space hole problem is converted into a pose calculation in a two-dimensional coordinate system, and the algorithm complexity is reduced. Finally, the pose of the positioning target is calculated according to the image, the pose freedom degree of the lifting appliance is obtained, the dependence of the human eye observation scheme is avoided, the measurement accuracy is improved, and the lifting appliance movement state is comprehensively monitored. Only a monocular camera and a control point target are needed in the above process, without high-cost sensors such as a laser radar and a binocular camera, and the equipment calibration requirement is low,

[0076] Moreover, compared with the binocular vision scheme, the binocular stereo measurement needs an overlap between two camera pictures, and only the overlap area is the positioning monitoring area. Therefore, if the same camera and the same focal length lens are installed at the same installation position, the monocular camera monitoring field of view provided in the embodiment is large, the effective pixels in imaging are more, a larger range scene can be monitored, the closer position can be monitored in the lifting hole alignment scene, and the alignment adjustment can be monitored. Because the effective pixels in the monocular camera imaging are more, the pixel proportion factor (mm / px) of the monitoring area imaging can be made smaller and more fine, and therefore the measurement accuracy of the embodiment is more accurate.

[0077] As shown in Figure 3 and Figure 4 , wherein, Figure 3 is a schematic diagram of a control point target in an embodiment of the application. Figure 4 is a schematic diagram of a control point target and a target local coordinate system thereof in the field of view of a monocular camera in an embodiment of the application.

[0078] Control point target 1 provides a local coordinate system for the target, providing a basic coordinate system and control point features for the positioning of the lifting device and lifting hole. Its pattern can be a white circle on a black background, with at least three white circles; alternatively, it can be other patterns, such as a QR code. In this embodiment, control point target 1 has a central circle at its center and several (e.g., four) control circles symmetrical about the center circle. The center point O of the central circle is defined as the center point, and the centers A, B, C, and D of the control circles are defined as control points, forming a control point set. Multi-point identification of control points can play a certain role in error suppression and resist ambient light interference. The reference target 11 and the positioning target 12 have the same pattern, only their placement differs.

[0079] like Figure 4 As shown, reference target 11 is installed on the beam 2 to be lifted. Its position is not strictly limited and can be flexibly set according to the local coordinate system of the reference target. Since reference target 11 is in contact with the top surface of the beam segment, the beam coordinate system can be established using the local coordinate system of the reference target without prior calibration. Step S1: When establishing the local coordinate system of the target based on control point target 1, the center O of the central circle on reference target 11 is defined as the origin of the local coordinate system; the direction of the line AB connecting the centers of the control circles on it is the transverse direction of the beam 2 to be lifted, and is defined as the origin of the local coordinate system of the reference target. X The axis; the direction of the line AD connecting the centers of the control circles is the longitudinal direction of the beam body 2 to be lifted, defined as the local coordinate system of the reference target. Y The Z-axis is the direction from point O perpendicular to the plane of reference target 11 outwards. For example, the black background border of reference target 11 can be installed parallel to the transverse edge of the beam segment 2 to be lifted.

[0080] The positioning target 12 is installed on the side wall of the longitudinal beam 31 of the lifting device 3. Its position is not strictly limited and can be flexibly set according to the local coordinate system of the positioning target. It only needs to be simply calibrated once before reuse, and the local coordinate system of the positioning target can be defined according to the on-site construction coordinate system. Similar to the establishment of the local coordinate system of the reference target, the center point O of the central circle on the positioning target 12 is defined as the origin of the local coordinate system; the direction of the side AB connecting the centers of the control circles on it is defined as the X-axis of the local coordinate system; the direction of the side AD connecting the centers of the control circles is parallel to the horizontal beam of the lifting device, representing the front-back direction of the lifting device 3, and is defined as the Y-axis of the local coordinate system; the Z-axis, perpendicular to the plane of the positioning target 12 and outward, is parallel to the horizontal beam of the lifting device, representing the left-right direction of the lifting device.

[0081] There are two positioning targets 12, which are respectively installed on the side walls of the two longitudinal beams 31 of the lifting device 3. For planar hole alignment, there are only two degrees of freedom (planar positioning). Therefore, as long as the two holes can be aligned, the lifting device can meet the hole alignment requirements. Furthermore, there is an anchor rod 32 anchored in front of the longitudinal beam 31. The positional distance between the longitudinal beam 31 and the monocular camera 4 may change, but the distance between the positioning target 12 and the anchor rod 32 anchored in front of the longitudinal beam 31 is relatively stable. The longitudinal beam 31 can be positioned by observing the positioning target 12 on the longitudinal beam 31, and the anchor rod 32 can be positioned by the distance value between the anchor rod 32 and the positioning target 12.

[0082] In step S2, when acquiring images containing control point targets 1 and lifting holes 20 using a monocular camera 4 mounted on the bridge erecting machine's lifting device 3, the monocular camera 4 observes vertically downwards the top of the beam to be lifted 2 and the longitudinal beam 31 area of ​​the lifting device 3, ensuring that at least two lifting holes 20, the reference target 11 near the lifting hole 20, and the positioning target 12 are covered in the field of view, and that the circles on the control point targets are fully imaged. This setup facilitates the relative alignment of the longitudinal beam 31 and the lifting holes 20 on the top of the beam to be lifted 2 according to the target local coordinate system designed by the control point targets 1, ultimately achieving coordinate system one for planar positioning of the anchor rod 32 and the lifting holes 20.

[0083] In one embodiment, the bridge erecting machine's hoisting hole monitoring system also includes a supplementary light 5 near the monocular camera 4, used to provide supplementary lighting for the hoisting hole 20 and the control point target 1.

[0084] The above scheme utilizes supplementary lighting to enhance the illumination of the observation area. The direction of the lighting is parallel to the optical axis of the monocular camera. This is used to supplement the lighting of the hoisting holes and control point targets when the ambient light is weak at night or on cloudy days, forming high-contrast images and resisting ambient light interference to a certain extent.

[0085] In one embodiment, step S3, calculating the pose of the target based on the image to obtain the pose degrees of freedom of the lifting device, includes:

[0086] Step S31: Calculate the local coordinates of the target in the reference target's local coordinate system based on the image. The following positioning coordinates ;

[0087] Step S32: Calculate the tilt vector of the positioning target. ;

[0088] Step S33: Calculate the center of the anchor bolt installed on the longitudinal beam of the lifting device in the local coordinate system of the reference target. Plane coordinates below ;

[0089] Step S34: Repeat the above steps to obtain the pose of each positioning target. and the corresponding anchor rod plane coordinates , i denotes the number of the positioning target / anchor rod;

[0090] Step S35, calculating the mean value of the of all positioning targets, as the elevation , vertical inclination , horizontal inclination , and rotation angle of the hoist respectively, so as to obtain the pose freedom of the hoist.

[0091] Through the above scheme, the positioning target position is mapped to the reference target coordinate system by calculating the positioning coordinates of the positioning target in the local coordinate system of the reference target, so as to eliminate camera distortion and perspective error; the hoist inclination is obtained by calculating the inclination vector; the positioning target and the hoist mechanical structure are associated by calculating the plane coordinates of the anchor rod, so as to convert the visual data into the anchor rod positioning parameters available in engineering; finally, the overall robustness is improved by suppressing single-target abnormal value through multi-target pose weighted average; the six-degree-of-freedom pose monitoring of the hoist of the bridge girder erection machine is realized through hierarchical pose solution and multi-target data fusion, and the translation and rotation deviation of the hoist is comprehensively covered.

[0092] In an embodiment, the step S31 of calculating the positioning coordinates of the positioning target in the local coordinate system of the reference target based on the image comprises:

[0093] Step S311, identifying the pixel coordinates of the center point and the control points of the control point target in the image; wherein the control point target has a center circle located at the center, and a plurality of control circles which are symmetric about the center of the center circle, the center of the center circle is defined as the center point, and the center of the control circle is defined as the control point.

[0094] Step S312, based on the pixel coordinates of the center point and the control points, using the PnP pose estimation algorithm to calculate the conversion matrix of the local coordinate system of the reference target to the camera coordinate system , the conversion matrix of the local coordinate system of the positioning target to the camera coordinate system .

[0095] Specifically, based on the pixel coordinates of the center point and the control points on the reference target in the local coordinate system of the reference target, the pixel coordinate corresponding relationship of the object point-image point matching, and the internal parameters of the monocular camera (such as focal length, principal point offset, distortion coefficient), the PnP (Perspective-n-Point) pose estimation algorithm is used to calculate the conversion matrix of the local coordinate system of the reference target to the camera coordinate system (abbreviated as coordinate system) , the conversion matrix of the local coordinate system of the positioning target​ transformation matrix to the camera coordinate system .

[0096] PnP (Perspective-n-Point) pose estimation algorithm is to solve the pose of the camera according to the given 3D space point coordinates, and its corresponding 2D projection point coordinates in the image and the intrinsic matrix, that is, the position and direction relative to the target object, and a balance between real-time and accuracy is achieved.

[0097] Step S313, calculate the local coordinate system of the positioning target to the translation transformation vector of the reference target local coordinate system . , the positioning coordinates of the positioning target in the reference target local coordinate system ;

[0098] The calculation formula is: .

[0099] In an embodiment, step S32, calculate the inclination vector of the positioning target comprises:

[0100] Step S321, calculate the rotation transformation matrix of the local coordinate system of the positioning target to the reference target local coordinate system .

[0101] The calculation formula is: ;

[0102] Step S322, calculate the inclination vector of the positioning target according to the rotation transformation matrix .

[0103] The calculation formula is: ;

[0104] ;

[0105] wherein, represents the third column vector of the rotation transformation matrix , represents the sign function that outputs 1 when the variable takes a negative number, and outputs 0 when the variable takes the rest of the number; represents the first element of the vector .

[0106] In an embodiment, step S33, calculate the planar coordinates of the anchor rod center installed on the sling longitudinal beam in the reference target local coordinate system ​​​​​​Comprising:

[0107] Step S331, obtaining the transverse distance from the anchor rod center to the positioning target center and the longitudinal distance ;

[0108] Step S332, calculating the planar coordinates of the anchor rod center in the reference target local coordinate system ; ;

[0109] The calculation formula is: ;

[0110] Wherein, is the positioning coordinates of the positioning target in the reference target local coordinate system .

[0111] In an embodiment, the bridge girder erection machine hoist hole monitoring method further comprises:

[0112] Step S4, calculating the planar coordinates of the hoist hole in the reference target local coordinate system based on the image as the positioning result of the hoist hole.

[0113] Step S4, calculating the planar coordinates of the hoist hole in the reference target local coordinate system based on the image as the positioning result of the hoist hole comprises:

[0114] Step S41, calculating the hoist hole center pixel coordinates in the image based on the image.

[0115] Specifically, taking one of the two hoist holes as an example, the YOLO target detection network is used to first coarsely position the hoist hole region in the monocular camera field of view, and the four elements of the target anchor box are outputted, and the upper left point pixel coordinates and the frame length and width of the hoist hole belonging to the image sub-region are determined. In each detected sub-region, an ellipse fitting is used to finely position the hoist hole center to obtain the hoist hole center pixel coordinates .

[0116] Step S42, identifying all control point pixel coordinates ,..., , n on the reference target in the image.

[0117] Step S43, constructing an over-determined equation based on the coordinates ,..., of all control points on the reference target in the reference target local coordinate system ;

[0118] Wherein, ;

[0119] .

[0120] Step S44, singular value decomposition (SVD) is performed, and the right singular vector corresponding to the minimum singular value is taken as the solution of . SVD decomposition is numerically stable, directly processes ill-conditioned matrices, and avoids the failure problem of traditional analytical methods when the control points are collinear.

[0121] Step S45, the coordinates of the center of the lifting hole in the local coordinate system of the reference target are calculated, and the plane coordinates are taken as the positioning results of the lifting hole.

[0122] The calculation formula is: .

[0123] Through the above scheme, high-precision image processing and overdetermined equation optimization are realized to achieve sub-millimeter-level positioning of the lifting hole in the local coordinate system of the target. Since the lifting hole and the reference target are coplanar, only the X-axis and Y-axis directions need to be calculated, and the Z-axis does not need to be considered, thereby reducing the algorithm complexity.

[0124] The above calculation process is completed by the edge computing terminal 6, which includes camera acquisition hardware interface, computing board, communication board and other structures to realize image data acquisition, image processing, lifting device-lifting hole relative pose solution and pose data transmission functions.

[0125] In an embodiment, the bridge crane lifting device hole monitoring method further comprises:

[0126] Step S5, the lifting device-lifting hole alignment control is performed based on the pose degrees of freedom of the lifting device.

[0127] Step S5, the lifting device-lifting hole alignment control based on the pose degrees of freedom of the lifting device comprises:

[0128] Step S51, the lifting device pose controller receives the elevation , vertical inclination , horizontal inclination , rotation angle and plane coordinates of the anchor rod output by the edge computing terminal 6.

[0129] Step S52, the lifting device pose controller performs PID control through the lifting device optimization objective function to drive the lifting device pose actuator to adjust the pose of the lifting device until the lifting device optimization objective function converges; the lifting device optimization objective function includes an elevation optimization objective function and a pose optimization objective function .

[0130] The elevation optimization objective function is represented as: ;

[0131] The attitude optimization objective function is represented as: .

[0132] wherein, represents the target elevation during the preset alignment adjustment.

[0133] Specifically, PID control is the most classic feedback control algorithm in the field of industrial control. Through the synergistic effect of the proportional (P), integral (I), and differential (D) three links, dynamic adjustment of system error is achieved. The principle is to calculate the control amount according to the error between the set value and the actual value, so as to realize high robustness control with a simple structure.

[0134] In step S53, the spreader pose controller performs PID control on the plane-to-hole optimization objective function, and drives the spreader pose actuator to adjust the plane position of the spreader and its anchor rod until the plane-to-hole optimization objective function converges.

[0135] The plane-to-hole optimization objective function is represented as: .

[0136] wherein, N is the number of positioning targets / anchor rods;

[0137] In step S54, the spreader pose actuator is driven to make the spreader fall until the anchor rods of the spreader are inserted into the lifting holes.

[0138] Through the above scheme, the multi-objective hierarchical PID control and the optimization objective function are driven to first adjust the attitude, then align the plane, and finally fall, avoiding damage to the lifting holes due to oblique insertion, so as to realize high-precision, self-adaptive, and full-automatic alignment of the spreader of the bridge girder and the lifting holes of the beam body in a more comprehensive way. The independent PID control optimizes the elevation and attitude respectively, and the optimization objective function is explicitly separated (to avoid coupling interference).

[0139] Specifically, the spreader pose controller 7 is located on the spreader 3. The spreader pose freedom degree mainly includes height and plane position, rotation angle, transverse inclination angle, and vertical inclination angle. The spreader pose controller 7 has an interface for communication with the edge computing terminal 6. The spreader pose controller 7 is configured to: receive the feedback information output by the edge computing terminal 6, based on the pose freedom degree of the spreader, and output a spreader pose adjustment instruction according to the characteristics of the spreader pose actuator 8.

[0140] The spreader pose actuator 8 is configured to: receive the spreader pose adjustment instruction output by the spreader pose controller, and adjust the spreader pose.

[0141] In an embodiment, the sling pose executor 8 comprises a hoisting trolley 81, a slewing adjusting mechanism 82, a transverse adjusting mechanism 83, and a vertical adjusting mechanism 84, which respectively receive the sling pose adjusting instructions output by the sling pose controller 7 to adjust the sling height and / or plane position, the slewing angle, the transverse inclination angle, and the vertical inclination angle.

[0142] As shown in Figure 2 the application also provides a single-camera-vision-based bridge girder erection machine sling hole monitoring system, which applies a single-camera-vision-based bridge girder erection machine sling hole monitoring method, which comprises the following steps:

[0143] a control point target 1 for establishing a target local coordinate system, the control point target 1 comprising a reference target 11 installed on the top of a to-be-lifted girder 2 having a lifting hole 20 and coplanar with the lifting hole 20, and a plurality of positioning targets 12 installed on a longitudinal beam 31 of a bridge girder erection machine sling 3, the target local coordinate system comprising a reference target local coordinate system corresponding to the reference target 11 and a positioning target local coordinate system corresponding to the positioning targets 12; ;

[0144] a single-camera 4 installed on the bridge girder erection machine sling 3 and configured to collect an image containing the control point target 1 and the lifting hole 20;

[0145] an edge computing terminal 6 configured to obtain the image, solve the pose of the positioning targets 12 according to the image to obtain the pose freedom degree of the sling 3, and output feedback information.

[0146] In an embodiment, the bridge girder erection machine sling hole monitoring system further comprises:

[0147] a sling pose controller 7 configured to receive the feedback information output by the edge computing terminal 6 and output sling pose adjusting instructions based on the pose freedom degree of the sling 3;

[0148] a sling pose executor 8 configured to receive the sling pose adjusting instructions output by the sling pose controller 7 and adjust the pose of the sling 5.

[0149] In an embodiment, the bridge girder erection machine sling hole monitoring system further comprises a light supplementing lamp 5 close to the single-camera 4 for supplementing light for the lifting hole 20 and the control point target 1.

[0150] In an embodiment, the sling pose executor 8 comprises a hoisting trolley 81, a slewing adjusting mechanism 82, a transverse adjusting mechanism 83, and a vertical adjusting mechanism 84, which respectively receive the sling pose adjusting instructions output by the sling pose controller 7 to adjust the sling height and / or plane position, the slewing angle, the transverse inclination angle, and the vertical inclination angle.

[0151] ​The parts of the bridge machine sling hole monitoring system have been described above and will not be described again here.

[0152] In the description of the present application, it should be noted that the terms "upper", "lower", and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. Unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixed connection, or detachable connection, or integral connection; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0153] It should be noted that in the present application, relational terms such as "first" and "second" and the like are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0154] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0155] The above is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications of these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for monitoring the alignment of a spreader of a bridge girder machine based on monocular machine vision, characterized in that, It comprises the following steps: A target local coordinate system is established according to a control point target; wherein the control point target comprises a reference target installed on a top of a to-be-hoisted beam body having a lifting hole and coplanar with the lifting hole, and a plurality of positioning targets installed on longitudinal beams of a bridge girder hoist; the target local coordinate system comprises a reference target local coordinate system corresponding to the reference target , and a positioning target local coordinate system corresponding to the positioning targets ​ acquiring an image containing the control point target and the hole by a monocular camera installed on the bridge machine hoist; solving the pose of the positioning target according to the image to obtain the pose freedom degree of the hoist; wherein, the step of solving the pose of the positioning target according to the image to obtain the pose freedom degree of the hoist comprises: calculating positioning coordinates of the positioning target in a local coordinate system of the reference target based on the image ;​ calculating an inclination vector of the positioning target ; calculating the planar coordinates of the center of the anchor rod mounted on the sling longitudinal beam in the local coordinate system of the reference target ;​ repeating the above steps to obtain the pose of each of the positioning targets and the corresponding anchor rod plane coordinates , i denotes the number of positioning targets / anchors the mean of all positioning targets is calculated as the elevation , vertical inclination , lateral inclination , rotation angle of the spreader, respectively, so as to obtain the pose freedom of the spreader.

2. The monocular machine vision based spreader pair hole monitoring method of claim 1, wherein, The local coordinate system of the positioning target based on the image is calculated in the local coordinate system of the reference target. The following positioning coordinates include: identifying the pixel coordinates of the center point and control points of the control point target in the image; wherein, the control point target has a center circle located at the center, and a plurality of control circles which are symmetric about the center of the center circle, the center of the center circle is defined as the center point, and the center of the control circle is defined as the control point; calculating a conversion matrix from the camera coordinate system to the positioning target local coordinate system a conversion matrix from the camera coordinate system to the positioning target local coordinate system a conversion matrix from the camera coordinate system to the positioning target local coordinate system a conversion matrix from the camera coordinate system to the positioning target local coordinate system a conversion matrix from the camera coordinate system to the positioning target local coordinate system computing the positioning target local coordinate system a translation transformation vector to the reference target local coordinate system elements as positioning coordinates of the positioning target in the reference target local coordinate system under​​ The calculation formula is: .

3. The monocular machine vision based spreader pair hole monitoring method of claim 2, wherein, calculating an inclination vector of the positioning target comprising: computing a rotation transformation matrix from the positioning target local coordinate system to the reference target local coordinate system ;​​ The calculation formula is: ; According to the rotation transformation matrix calculating an inclination vector of the positioning target ; The calculation formula is: ; ； wherein denotes the third column column vector of the rotation matrix denotes the third column column vector of the rotation matrix denotes the sign function which outputs 1 if the variable takes a negative number and 0 if the variable takes any other value; denotes the first element of the vector denotes the first element of the vector 4. The monocular machine vision based spreader pair hole monitoring method of claim 2, wherein, The bridge machine hoist hole monitoring method further comprises: calculating, based on the image, planar coordinates of the hole in a local coordinate system of the reference target as a positioning result of the hole. The calculation of the hanging hole in the local coordinate system of the reference target based on the image The planar coordinates below, as the positioning result of the lifting hole, include: calculating, based on the image, the hole center pixel coordinates in the image ; identifying all control point pixel coordinates on the reference target in the image ,..., , n is the number of control points; based on the coordinates of all control points on the reference target the coordinates of all control points on the reference target ,..., , constructing overdetermined equations ; wherein ; ； Singular value decomposition is performed, and the right singular vector corresponding to the minimum singular value is taken as the solution of the equation calculating the coordinates of the center of the eyelet in the local coordinate system of the reference target with the following formula for the plane coordinates as a result of the positioning of the eyelet The calculation formula is: .

5. The monocular machine vision based spreader pair hole monitoring method of claim 1, wherein, The bridge machine hoist hole monitoring method further comprises: controlling the hoist-hole alignment based on the pose freedom degree of the hoist; The step of controlling the hoist-hole alignment based on the pose freedom degree of the hoist comprises: a hoist pose controller receives an elevation of the hoist , a vertical tilt angle , a lateral tilt angle , a rotation angle , and planar coordinates of the anchor rod ; The spreader pose controller drives the spreader pose actuator to adjust the pose of the spreader until the spreader optimization objective function converges through PID control of the spreader optimization objective function; the spreader optimization objective function includes an elevation optimization objective function and a pose optimization objective function ; The elevation optimization objective function is expressed as: ; The pose optimization objective function is represented as: ; wherein, represents a target elevation when preset alignment adjustment is performed; The hoist pose controller performs PID control on the plane hole optimization objective function to drive the hoist pose actuator to adjust the plane position of the hoist and its anchor rod until the plane hole optimization objective function converges; The planar pair-hole optimization objective function is expressed as: ; wherein, N n is the number of positioning targets / shank anchors; driving the hoist pose actuator to make the hoist fall until the anchor rods of the hoist are all inserted into the holes.

6. A single-vision-machine-vision-based monitoring system for the alignment of the bridge girder hanger of a bridge girder launcher, applying the single-vision-machine-vision-based monitoring method for the alignment of the bridge girder hanger of a bridge girder launcher according to any one of claims 1 to 5, characterized in that, It comprises: A control point target (1) for establishing a target local coordinate system, the control point target (1) comprising a reference target (11) mounted on the top of a to-be-hung girder body (2) having a lifting hole (20) and coplanar with the lifting hole (20), and a plurality of positioning targets (12) mounted on a longitudinal beam (31) of a bridge girder erection machine lifting device (3), the target local coordinate system comprising a reference target local coordinate system corresponding to the reference target (11) , and a positioning target local coordinate system corresponding to the positioning targets (12) ​ a monocular camera (4) installed on the bridge machine hoist (3) and configured to acquire an image containing the control point target (1) and the hole (20); an edge computing terminal (6) configured to acquire the image, solve the pose of the positioning target (12) according to the image to obtain the pose freedom degree of the hoist (3), and output feedback information; wherein, the step of solving the pose of the positioning target according to the image to obtain the pose freedom degree of the hoist comprises: calculating positioning coordinates of the positioning target in a local coordinate system of the reference target based on the image ;​ calculating an inclination vector of the positioning target ; calculating the planar coordinates of the center of the anchor rod mounted on the sling longitudinal beam in the local coordinate system of the reference target ;​ repeating the above steps to obtain the pose of each of the positioning targets and the corresponding anchor rod planar coordinates , i denotes the number of positioning targets / anchors the mean of all positioning targets is calculated as the elevation , vertical inclination , lateral inclination , rotation angle of the hoist, respectively, so as to obtain the pose freedom of the hoist.

7. The monocular machine vision-based spreader-to-portal monitoring system of claim 6, wherein, The bridge machine hoist hole monitoring system further comprises: a hoist pose controller (7) configured to receive the feedback information output by the edge computing terminal (6), and output hoist pose adjustment instructions based on the pose freedom degree of the hoist (3); a hoist pose actuator (8) configured to receive the hoist pose adjustment instructions output by the hoist pose controller (7) to adjust the pose of the hoist (3).

8. The monocular machine vision-based spreader-to-socket monitoring system of claim 6, wherein, The bridge machine hoist hole monitoring system further comprises a light supplement lamp (5) close to the monocular camera (4) for supplementing light for the hole (20) and the control point target (1).

9. The monocular machine vision-based spreader pair hole monitoring system of claim 7, wherein, The hoist pose actuator (8) comprises a hoist trolley (81), a rotation adjusting mechanism (82), a transverse adjusting mechanism (83), and a vertical adjusting mechanism (84), which respectively receive the hoist pose adjustment instructions output by the hoist pose controller (7) to adjust the height and / or plane position, rotation angle, transverse inclination angle, and vertical inclination angle of the hoist (3).

Images