Bridge erecting machine deformation monitoring and early warning system and method based on computer vision

Abstract

The invention particularly relates to a bridge erecting machine deformation monitoring and early warning system and method based on computer vision, and relates to the technical field of engineering machinery safety monitoring. A distortion correction module; a target detection module; and a deformation calculation module. According to the invention, the ArUco target is combined with a high-resolution industrial camera to realize non-contact measurement, lens distortion is eliminated by using an OpenCV camera calibration algorithm through a distortion correction module, and the precision of displacement and inclination angle calculation is greatly improved by combining a sub-pixel-level angular point positioning technology; the target detection module is used for quickly positioning the target, and the deformation calculation module is used for correcting the motion interference of the camera by utilizing the reference target, so that the actual physical displacement of a measuring point can be output in real time; the real-time performance and high precision are combined, fine deformation of the bridge girder erection machine in the girder erection process can be captured in time, potential safety hazards caused by lagging monitoring are avoided, and accurate data support is provided for construction safety.

Description

Technical Field

[0001] This invention relates to the field of engineering machinery safety monitoring technology, and in particular to a deformation monitoring and early warning system and method for bridge erecting machines based on computer vision. Background Technology

[0002] A bridge erecting machine is a type of lifting equipment used to place prefabricated bridge beams onto bridge piers. The construction conditions for bridge erecting machines are quite unique, with stringent safety requirements. During operation, the outriggers of the bridge erecting machine are fixed to the bridge piers, and then the main body of the machine is moved via a traveling system.

[0003] Currently, widely used monitoring methods for bridge erecting machines include contact sensor monitoring and total station monitoring. By installing strain gauges, tilt sensors, or displacement gauges on key components such as the main beam and outriggers of the bridge erecting machine, structural response data can be collected in real time. However, the vibration environment in the field leads to a high failure rate for contact sensors. When the bridge erecting machine is performing a step-by-step jacking operation, the sensors cannot distinguish between rigid body motion caused by support point displacement and the elastic deformation of the structure itself. Using a high-precision total station to periodically measure preset targets on the bridge erecting machine avoids some of the drawbacks of contact sensors, but in practical applications, frequent station changes may be necessary as the bridge erecting machine moves, introducing unnecessary errors, and each measurement is time-consuming.

[0004] Existing monitoring methods lack ease of use and reliability in dynamic construction scenarios involving bridge erecting machines. A new monitoring method with adaptive dynamic benchmarks needs to be developed to achieve accurate perception and proactive safety warnings of the bridge erecting machine's structural status throughout the entire process. Summary of the Invention

[0005] The purpose of this invention is to provide a computer vision-based deformation monitoring and early warning system and method for bridge erecting machines in order to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A computer vision-based bridge erecting machine deformation monitoring and early warning system includes:

[0008] ArUco target deployment and image acquisition module configuration: After each ArUco target is installed in its corresponding preset position, the corresponding image acquisition operation is performed.

[0009] The distortion correction module is configured to use the corner information marked by ArUco for camera calibration, calculate the intrinsic and extrinsic parameter matrices and distortion parameters, and perform image filtering after eliminating lens distortion.

[0010] The target detection module is configured to use a deep learning-based ArUco marker detection model to locate the bounding box of ArUco targets in the image;

[0011] The deformation calculation module is configured to calculate the tilt angle, displacement, and scale factor of the ArUco reference point target based on the acquired image, thereby calculating the actual physical displacement of the ArUco target at the measuring point.

[0012] Preferably, the ArUco target deployment and image acquisition module specifically includes:

[0013] The standard ArUco marker library is selected, and each target is assigned a unique ID. During installation, ensure that the target plane is orthogonal to the axis of the measuring point.

[0014] The image acquisition module consists of an industrial camera, fixed on a steel base plate. The x-axis is along the direction of the main beam of the bridge erecting machine, the z-axis is along the direction of the plumb bob, and the y-axis is orthogonal to the x-axis and z-axis.

[0015] The image acquisition module is installed on the outside of the main beam directly above the middle leg of the bridge erecting machine to capture images of the ArUco reference point target installed below the middle leg, as well as ArUco measuring point target images on other beams and legs.

[0016] Preferably, the distortion correction module specifically includes the following steps:

[0017] Identify ArUco markers in the image and output the corner pixel coordinates of each marker. Let the corner pixel coordinates of the marker with ID k be:

[0018] P k ={(u1,v1),(u2,v2),(u3,v3),(u4,v4)}

[0019] Where (u1, v1) represents the pixel position in the image coordinate system;

[0020] Fix the world coordinate system to the plane of the ArUco calibration plate; calculate the world coordinates of the corner points based on the actual physical dimensions and side length L of each marker.

[0021] The world coordinates of the corner point of the k-th marker are:

[0022] P={(0,0,0),(L,0,0),(L,L,0),(0,L,0)}

[0023] And convert to homogeneous coordinates:

[0024]

[0025] Initialize the camera parameters:

[0026] The projection relationship between pixel coordinates and world coordinates is obtained:

[0027]

[0028] in,

[0029] K is the intrinsic parameter matrix: f x f y c is the focal length. x c y Main point;

[0030] R is the rotation matrix;

[0031] t is the translation vector;

[0032] The parameters are optimized to obtain the corrected image.

[0033] Preferably, the target detection module specifically includes:

[0034] The acquired images undergo preprocessing including grayscale conversion, adaptive histogram equalization, and Gaussian filtering for noise reduction.

[0035] ArUco marker detection is performed, including adaptive threshold segmentation, preserving only quadrilateral contours, and perspective transformation to extract marker regions.

[0036] Preferably, the method further includes corner point positioning:

[0037] Initial location of Harris corner: R = det(M) - k·(trace(M)) 2 ;

[0038] in

[0039] Through iterative subpixel optimization:

[0040] Iterative update: x n+1 =x n +Δx,y n+1 =y n +Δy.

[0041] Preferably, the further step includes coordinate system mapping:

[0042] Mark center calculation:

[0043] Inclination calculation:

[0044] in,

[0045] n is the unit axis of rotation;

[0046] θ is the rotation angle;

[0047] The tilt angle is derived based on the rotation matrix and its trace, including:

[0048] Rotation matrix: R = cosθ·I + (1-cosθ)nn T +sinθ·[n] x ;

[0049] in,

[0050] I is a 3×3 identity matrix;

[0051] [n] x Here are the antisymmetric matrices for the rotation axes:

[0052] based on:

[0053]

[0054] Where tr(R) is the trace of the matrix.

[0055] Preferably, the deformation calculation module specifically includes:

[0056] Using a reference target, the scaling factor is determined, and the physical displacement is obtained by multiplying the pixel displacement by the scaling factor.

[0057] Using the target image at the reference point, the displacement and tilt angle of the image acquisition module are calculated in reverse, the physical displacement of the target at the measurement point is corrected, and the interference of the camera's own motion is eliminated by combining the rotation matrix and displacement inverse calculation logic to obtain the actual physical displacement of the target at the measurement point.

[0058] Preferably, the early warning module is configured to match the corresponding early warning level and perform corresponding processing based on the comparison between the actual physical displacement of the measuring point ArUco target and a preset threshold, wherein the early warning level includes level one, level two and level three.

[0059] A computer vision-based method for monitoring and early warning of deformation in bridge erecting machines includes the following steps:

[0060] Target deployment and image acquisition: The standard ArUco marker library is selected, and a unique ID is assigned to each target; the number and position of measuring point targets and reference targets are adjusted according to engineering requirements; the image acquisition module consists of an industrial camera, which is fixed on a steel base plate and installed as a whole on the outside of the main beam directly above the middle support leg;

[0061] Image distortion correction: Identify ArUco markers in the image, extract the pixel coordinates of the four corner points of each marker, calculate the world coordinates of the corner points and convert them to homogeneous coordinates; then initialize camera parameters, establish the projection relationship between pixel coordinates and world coordinates, and describe the mapping law; minimize the reprojection error, solve the intrinsic parameter matrix, distortion coefficients and extrinsic parameters; calculate the parameters to correct the image and eliminate lens distortion;

[0062] Target detection and precise corner localization: The acquired and corrected images are preprocessed, and ArUco marker detection is performed to extract the marker regions;

[0063] Deformation calculation: Based on the located corner point information, calculate the center coordinates and tilt angle of the ArUco marker: Combine the rotation matrix and displacement inverse calculation logic to correct the physical displacement of the measuring point target, eliminate the interference of the camera's own motion, and finally obtain the actual physical displacement of the measuring point target;

[0064] Early warning processing: Based on the comparison between the actual physical displacement of the ArUco target at the measuring point and the preset threshold, the corresponding early warning level is matched and the corresponding processing is performed.

[0065] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0066] 1. This invention achieves non-contact measurement by combining an ArUco target with a high-resolution industrial camera. A distortion correction module utilizes OpenCV's camera calibration algorithm to eliminate lens distortion, and combined with sub-pixel corner positioning technology, significantly improves the accuracy of displacement and tilt angle calculations. The target detection module quickly locates the target, and the deformation calculation module uses a reference target to correct for camera motion interference, enabling real-time output of the actual physical displacement of the measurement point. This combination of real-time performance and high precision allows for timely capture of subtle deformations of the bridge erecting machine during beam erection, avoiding safety hazards caused by delayed monitoring and providing accurate data support for construction safety.

[0067] 2. This invention allows for adjustments to the number and location of ArUco targets based on actual engineering needs. The layout of the three cameras can cover key components such as the main beam and outriggers of the bridge erecting machine, adapting to the monitoring requirements of different models of bridge erecting machines. The early warning module adopts a three-level hierarchical mechanism, automatically matching the safety status through preset thresholds, achieving full automation from data acquisition to risk response. This intelligent early warning not only reduces the subjectivity and lag of manual monitoring but also allows managers to take targeted measures based on the early warning level, reducing labor costs and improving the precision and efficiency of engineering management. It is particularly suitable for the safety management of bridge erecting machines in complex construction environments. Attached Figure Description

[0068] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0069] Figure 1 This is a flowchart of the method of the present invention;

[0070] Figure 2 This is a layout diagram of the bridge erecting machine and related monitoring equipment of the present invention;

[0071] Figure 3 This is a schematic diagram of the camera module of the present invention;

[0072] Figure 4 This is a diagram of the ArUco target of the present invention. Detailed Implementation

[0073] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0074] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0075] Example 1

[0076] Its specific implementation method is combined with the appendix Figure 1 Appendix Figure 2 Appendix Figure 3 and attached Figure 4 Please provide a detailed explanation.

[0077] Appendix Figure 1 This invention provides a structural block diagram of a bridge erecting machine deformation monitoring and early warning system based on computer vision. The diagram shows the connection relationship between the ArUco target deployment and image acquisition module and the deformation calculation module, and marks the main functional interaction flow of each module.

[0078] In this embodiment, it includes:

[0079] ArUco target deployment and image acquisition module configuration: After each ArUco target is installed in its corresponding preset position, the corresponding image acquisition operation is performed.

[0080] Specifically, it includes:

[0081] The standard ArUco marker library is selected, and each target is assigned a unique ID for easy subsequent calculations. Each marker measures 10cm x 10cm and uses a rigid aluminum alloy backplate. During installation, ensure the target plane is orthogonal to the axis of the measuring point.

[0082] The positions of the measuring target and the reference target can be referenced. Figure 2 The number and location of targets can be adjusted according to actual engineering needs.

[0083] Image acquisition module such as Figure 3 The image shows a system consisting of three industrial cameras fixed to a steel base plate. The x-axis runs along the main beam of the bridge erecting machine, the z-axis runs along the plumb line, and the y-axis is orthogonal to both the x-axis and z-axis.

[0084] Two of the cameras have optical axes parallel to the x-axis and in opposite directions; the optical axis of one camera points to the ArUco reference point target below the middle support leg and is parallel to the z-axis.

[0085] The image acquisition module is installed on the outside of the main beam directly above the middle leg of the bridge erecting machine. It is used to capture images of the ArUco reference point target installed below the middle leg, as well as ArUco measuring point target images on other beams and legs. The camera supports high resolution (such as 1920×1080 and above) and autofocus, and transmits image data via USB or network.

[0086] During the bridge erection process, images of the target are continuously captured, and clear, unobstructed images are selected and transmitted to the target detection module.

[0087] The distortion correction module is configured to use the corner information marked by ArUco for camera calibration, calculate the intrinsic and extrinsic parameter matrices and distortion parameters, and perform image filtering after eliminating lens distortion.

[0088] Specifically, the following steps are included:

[0089] ArUco code detection and corner extraction:

[0090] Use OpenCV's cv2.aruco.detectMarkers() function to identify ArUco markers in an image and output the pixel coordinates of the corner points of each marker (4 corner points per marker); let the pixel coordinates of the corner points of the marker with ID k be:

[0091] P k ={(u1,v1),(u2,v2),(u3,v3),(u4,v4)}

[0092] Where (u1, v1) represents the pixel position in the image coordinate system;

[0093] Fix the world coordinate system to the plane of the ArUco calibration plate; calculate the world coordinates of the corner points based on the actual physical dimensions and side length L of each marker.

[0094] The world coordinates of the corner point of the k-th marker are:

[0095] P k ={(0,0,0),(L,0,0),(L,L,0),(0,L,0)}

[0096] And convert to homogeneous coordinates:

[0097]

[0098] Initialize the camera parameters:

[0099] The projection relationship between pixel coordinates and world coordinates is obtained:

[0100]

[0101] in,

[0102] K is the intrinsic parameter matrix: f x f y c is the focal length. x c y Main point;

[0103] R is the rotation matrix;

[0104] t is the translation vector;

[0105] Radial and tangential distortion correction equations:

[0106] x distorted =x(1+k1r) 2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 )

[0107] y distorted =y(1+k1r) 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy

[0108] Where, r 2 =x 2 +y 2 The distortion coefficient is d = [k1, k2, p1, p2, k3].

[0109] Optimize the solution for the parameters:

[0110] Minimize reprojection error calculation:

[0111]

[0112] in,

[0113] N is the number of tags;

[0114] Project(·) is the projection function;

[0115] Optimization algorithm: Levenberg-Marquardt iterative method (implemented in OpenCV using cv2.calibrateCamera());

[0116] After solving, the output includes the intrinsic parameter matrix K, distortion coefficients d, and extrinsic parameter R. k ,t k ;

[0117] Use the cv2.undistort() function to apply the calculated K and d to obtain the corrected image:

[0118] I corrected =undistor(I input ,K,d).

[0119] The target detection module is configured as a deep learning-based ArUco marker detection model (such as OpenCV's ArUco library or a YOLOv5 custom model) to quickly locate the bounding box of ArUco targets in the image.

[0120] Specifically, it includes:

[0121] The acquired images undergo preprocessing including grayscale conversion, adaptive histogram equalization, and Gaussian filtering for noise reduction.

[0122] Grayscale conversion to reduce computational complexity: I grop (x,y)=0.299R+0.587G+0.114B;

[0123] Adaptive histogram equalization includes: segmenting the image into 8×8 blocks; performing histogram equalization on each block;

[0124] Gaussian filter noise reduction:

[0125] ArUco marker detection is performed, including adaptive threshold segmentation, preservation of quadrilateral contours only, and perspective transformation to extract marker regions;

[0126] Adaptive threshold segmentation: Threshold = μlocal -k·σ local ;

[0127] Where μ local For local mean, σ local The standard deviation is given, and k is 0.7.

[0128] Only the quadrilateral outline is retained (meeting the conditions):

[0129] The marked region is extracted using perspective transformation. The marked region is obtained using the cv2.getPerspectiveTransform and cv2.warpPerspective functions of OpenCV.

[0130] It also includes subpixel-level corner positioning:

[0131] Initial location of Harris corner: R = det(M) - k·(trace(M)) 2 ;

[0132] in

[0133] Through iterative subpixel optimization:

[0134] Iterative update: x n+1 =x n +Δx,y n+1 =y n +Δy.

[0135] This also includes coordinate system mapping:

[0136] Mark center calculation:

[0137] Inclination calculation:

[0138] in,

[0139] n is the unit axis of rotation;

[0140] θ is the rotation angle;

[0141] The tilt angle is derived based on the rotation matrix and its trace, including:

[0142] Rotation matrix: R = cosθ·I + (1-cosθ)nn T +sinθ·[n] x ;

[0143] in,

[0144] I is a 3×3 identity matrix;

[0145] [n]x Here are the antisymmetric matrices for the rotation axes:

[0146] based on:

[0147]

[0148] Where tr(R) is the trace of the matrix; tr(R) = r 11 +r 22 +r 33 .

[0149] The deformation calculation module is configured to calculate the tilt angle, displacement, and scale factor of the ArUco reference point target based on the acquired image, thereby calculating the actual physical displacement of the ArUco target at the measuring point.

[0150] Specifically, it includes:

[0151] Using a reference target, the scaling factor is determined, and the physical displacement is obtained by multiplying the pixel displacement by the scaling factor.

[0152] Using the target image at the reference point, the displacement and tilt angle of the image acquisition module are calculated in reverse, the physical displacement of the target at the measurement point is corrected, and the interference of the camera's own motion is eliminated by combining the rotation matrix and displacement inverse calculation logic to obtain the actual physical displacement of the target at the measurement point.

[0153] The early warning module is configured to match the corresponding early warning level and perform corresponding processing based on the comparison between the actual physical displacement of the ArUco target at the measuring point and the preset threshold. The early warning levels include Level 1, Level 2 and Level 3.

[0154] When the actual physical displacement of the target position at each measuring point is less than or equal to 60% of the preset threshold, the warning level is level one, and it is marked as a safe state (no alarm in data recording).

[0155] When the actual physical displacement of the target position at each measuring point is less than or equal to 80% of the preset threshold and greater than 60% of the preset threshold, the warning level is level two, and a yellow warning is triggered (mobile APP notification).

[0156] When the actual physical displacement of the target position at any measuring point is greater than 80% of the preset threshold, the warning level is level three, triggering a red alert alarm (audio-visual alarm).

[0157] Example 2

[0158] Please see the appendix Figure 1 A computer vision-based method for monitoring and early warning of deformation in bridge erecting machines includes the following components:

[0159] Target deployment and image acquisition: The standard ArUco marker library is selected, and each target is assigned a unique ID. Targets are 10cm x 10cm in size and fixed to a rigid aluminum alloy backplate. During installation, the target plane is ensured to be orthogonal to the axis of the measuring point. The number and position of the measuring point targets and the reference target are adjusted according to engineering requirements, with the reference target located below the middle support leg. The image acquisition module consists of three industrial cameras, fixed to a steel base plate. The x-axis is set along the direction of the main beam of the bridge erecting machine, the z-axis is along the direction of the plumb bob, and the y-axis is orthogonal to the x and z axes. Two cameras have optical axes parallel to the x-axis and in opposite directions, while one camera's optical axis points to the reference target below the middle support leg and is parallel to the z-axis. The entire module is installed on the outside of the main beam directly above the middle support leg. The cameras support resolutions of 1920×1080 and above and autofocus. Data is transmitted via USB or network, and target images are continuously captured during the bridge erection process. Clear, unobstructed images are selected and transmitted to subsequent steps.

[0160] Image distortion correction: The OpenCV function `cv2.aruco.detectMarkers()` is used to identify ArUco markers in the image, extracting the pixel coordinates of the four corner points of each marker. The world coordinate system is fixed on the ArUco calibration plate plane, and the world coordinates of the corner points are calculated and converted to homogeneous coordinates based on the actual side length of the marker. Next, camera parameters are initialized, and the projection relationship between pixel coordinates and world coordinates is established. The mapping law is described by the intrinsic matrix, rotation matrix, translation vector, and radial and tangential distortion correction equations. The Levenberg-Marquardt iterative method (implemented by `cv2.calibrateCamera()`) is used to minimize the reprojection error, solving for the intrinsic matrix, distortion coefficients, and extrinsic parameters. Finally, the `cv2.undistort()` function is used to apply the calculated parameters to correct the image and eliminate lens distortion.

[0161] Target detection and precise corner localization: The acquired and corrected images are preprocessed, including grayscale conversion to reduce computational complexity, segmentation into 8×8 blocks for adaptive histogram equalization to enhance contrast, and Gaussian filtering for noise reduction. ArUco marker detection is then performed, first using adaptive threshold segmentation (based on local mean and standard deviation, with a coefficient of 0.7), then preserving the quadrilateral contours, and finally extracting the marker regions using the cv2.getPerspectiveTransform and cv2.warpPerspective functions. Based on this, initial localization is performed using the Harris corner algorithm, followed by iterative optimization to achieve sub-pixel level corner localization, improving detection accuracy.

[0162] Deformation Calculation: Based on the located corner point information, the center coordinates and tilt angle of the ArUco marker are calculated. The tilt angle is derived through a rotation matrix and its trace, where the rotation matrix is ​​determined by the unit rotation axis and rotation angle, and the antisymmetric matrix of the rotation axis is used to assist in the calculation. Simultaneously, the scale factor is determined using the ArUco reference point target below the middle outrigger, converting pixel displacement into physical displacement. The displacement and tilt angle of the image acquisition module are inversely calculated using the reference point target image. Combined with the rotation matrix and displacement inverse calculation logic, the physical displacement of the measuring point target is corrected, eliminating interference from the camera's own motion, and finally obtaining the actual physical displacement of the measuring point target.

[0163] Early warning processing: Based on the comparison between the actual physical displacement of the ArUco target at the measuring point and the preset threshold, the corresponding early warning level is matched and the corresponding processing is performed.

[0164] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0165] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0166] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0167] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0168] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0171] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0172] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0173] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A computer vision-based deformation monitoring and early warning system for bridge erecting machines, characterized in that, include: ArUco target deployment and image acquisition module configuration: After each ArUco target is installed in its corresponding preset position, the corresponding image acquisition operation is performed. The distortion correction module is configured to use the corner information marked by ArUco for camera calibration, calculate the intrinsic and extrinsic parameter matrices and distortion parameters, and perform image filtering after eliminating lens distortion. The target detection module is configured to use a deep learning-based ArUco marker detection model to locate the bounding box of ArUco targets in the image; The deformation calculation module is configured to calculate the tilt angle, displacement, and scale factor of the ArUco reference point target based on the acquired image, thereby calculating the actual physical displacement of the ArUco target at the measuring point.

2. The deformation monitoring and early warning system for bridge erecting machines based on computer vision according to claim 1, characterized in that, The ArUco target deployment and image acquisition module specifically includes: The standard ArUco marker library is selected, and each target is assigned a unique ID. During installation, ensure that the target plane is orthogonal to the axis of the measuring point. The image acquisition module consists of an industrial camera, fixed on a steel base plate. The x-axis is along the direction of the main beam of the bridge erecting machine, the z-axis is along the direction of the plumb bob, and the y-axis is orthogonal to the x-axis and z-axis. The image acquisition module is installed on the outside of the main beam directly above the middle leg of the bridge erecting machine to capture images of the ArUco reference point target installed below the middle leg, as well as ArUco measuring point target images on other beams and legs.

3. The deformation monitoring and early warning system for bridge erecting machines based on computer vision according to claim 2, characterized in that, The distortion correction module includes the following steps: Identify ArUco markers in the image and output the corner pixel coordinates of each marker. Let the corner pixel coordinates of the marker with ID k be: P k ={(u1,v1),(u2,v2),(u3,v3),(u4,v4)} Where (u1, v1) represents the pixel position in the image coordinate system; Fix the world coordinate system to the plane of the ArUco calibration plate; calculate the world coordinates of the corner points based on the actual physical dimensions and side length L of each marker. The world coordinates of the corner point of the k-th marker are: P k <{(0,0,0),(L,0,0),(L,L,0),(0,L,0)} And convert to homogeneous coordinates: Initialize the camera parameters: The projection relationship between pixel coordinates and world coordinates is obtained: in, K is the intrinsic parameter matrix: f x f y c is the focal length. x c y Main point; R is the rotation matrix; t is the translation vector; The parameters are optimized to obtain the corrected image.

4. The deformation monitoring and early warning system for bridge erecting machines based on computer vision according to claim 1, characterized in that, The target detection module specifically includes: The acquired images undergo preprocessing including grayscale conversion, adaptive histogram equalization, and Gaussian filtering for noise reduction. ArUco marker detection is performed, including adaptive threshold segmentation, preserving only quadrilateral contours, and perspective transformation to extract marker regions.

5. The deformation monitoring and early warning system for a bridge erecting machine based on computer vision according to claim 4, characterized in that, It also includes corner point positioning: Initial location of Harris corner: R = det(M) - k·(trace(M)) 2 ; in Through iterative subpixel optimization: Iterative update: x n+1 =x n +Δx,y n+1 =y n +Δy.

6. The deformation monitoring and early warning system for a bridge erecting machine based on computer vision according to claim 5, characterized in that, This also includes coordinate system mapping: Mark center calculation: Inclination calculation: in, n is the unit axis of rotation; θ is the rotation angle; The tilt angle is derived based on the rotation matrix and its trace, including: Rotation matrix: R = cosθ·I + (1-cosθ)nn T +sinθ·[n] x ; in, I is a 3×3 identity matrix; [n] x Here are the antisymmetric matrices for the rotation axes: based on: Where tr(R) is the trace of the matrix.

7. The deformation monitoring and early warning system for bridge erecting machines based on computer vision according to claim 1, characterized in that, The deformation calculation module specifically includes: Using a reference target, the scaling factor is determined, and the physical displacement is obtained by multiplying the pixel displacement by the scaling factor. Using the target image at the reference point, the displacement and tilt angle of the image acquisition module are calculated in reverse, the physical displacement of the target at the measurement point is corrected, and the interference of the camera's own motion is eliminated by combining the rotation matrix and displacement inverse calculation logic to obtain the actual physical displacement of the target at the measurement point.

8. The deformation monitoring and early warning system for a bridge erecting machine based on computer vision according to claim 1, characterized in that, The early warning module is configured to match the corresponding early warning level and perform corresponding processing based on the comparison between the actual physical displacement of the ArUco target at the measuring point and the preset threshold. The early warning levels include Level 1, Level 2 and Level 3.

9. A computer vision-based method for monitoring and early warning of bridge erection machine deformation, as described in any one of claims 1-8, characterized in that... Includes the following steps: Target deployment and image acquisition: The standard ArUco marker library is selected, and a unique ID is assigned to each target; the number and position of measuring point targets and reference targets are adjusted according to engineering requirements; the image acquisition module consists of an industrial camera, which is fixed on a steel base plate and installed as a whole on the outside of the main beam directly above the middle support leg; Image distortion correction: Identify ArUco markers in the image, extract the pixel coordinates of the four corner points of each marker, calculate the world coordinates of the corner points and convert them into homogeneous coordinates; then initialize the camera parameters, establish the projection relationship between pixel coordinates and world coordinates, and describe the mapping law; minimize the reprojection error, and solve for the intrinsic parameter matrix, distortion coefficients and extrinsic parameters; The parameters are calculated to correct the image and eliminate lens distortion; Target detection and precise corner localization: The acquired and corrected images are preprocessed, and ArUco marker detection is performed to extract the marker regions; Deformation calculation: Based on the located corner point information, calculate the center coordinates and tilt angle of the ArUco marker: Combine the rotation matrix and displacement inverse calculation logic to correct the physical displacement of the measuring point target, eliminate the interference of the camera's own motion, and finally obtain the actual physical displacement of the measuring point target; Early warning processing: Based on the comparison between the actual physical displacement of the ArUco target at the measuring point and the preset threshold, the corresponding early warning level is matched and the corresponding processing is performed.

Images