A method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV)

Abstract

A structural deformation measurement method based on a cross-laser reference and a UAV is disclosed. The method employs a cross-laser emitting device to project a cross-laser pattern onto a target as a static reference. A UAV is used to capture close-range images of the measurement area. Based on the relative positional changes between the target and the projected pattern, non-contact, high-precision displacement and rotation measurements are achieved. The method includes: A. Deploying a structural deformation measurement system based on a cross-laser reference and a UAV in the target area and acquiring video data; B. Using YOLOv11 to identify the target in the video image and extracting the region of interest (ROI); C. Extracting the precise positions of the visual target and the cross-laser line through preliminary and precise two-stage positioning; D. Analyzing the relative positional changes between the target and the cross-laser line to calculate the target's displacement and rotational response. This invention's measurement method, by introducing a static laser reference and analyzing the relative changes between the target and the laser pattern, achieves simultaneous two-dimensional displacement and rotation measurement, offering the advantages of non-contact and high-precision measurement.

Description

Technical Field

[0001] This invention relates to a method for measuring structural deformation based on a cross laser reference and an unmanned aerial vehicle (UAV), belonging to the field of structural inspection. Background Technology

[0002] Structures (bridges, buildings, etc.) are subjected to various factors during their service life, including traffic loads, wind loads, and temperature changes. The displacement or rotation response of their critical components is an important parameter for assessing the structural health and safety performance. Therefore, developing efficient and accurate methods for measuring structural response is of great significance for ensuring the safe service of structures.

[0003] Existing structural response measurements can be categorized into contact and non-contact methods. Contact methods primarily utilize devices such as LVDT displacement sensors, dial gauges, and inclinometers. These methods typically require sensor installation at the measurement points, which presents challenges in areas difficult for personnel to access (such as the base of bridges or the top of high piers), leading to implementation difficulties and high costs. Non-contact methods mainly include total stations and the Global Positioning System (GPS). Total stations offer high static measurement accuracy but struggle with continuous dynamic measurement of targets. GPS technology enables remote monitoring, but its stability is insufficient in environments with blocked satellite signals, making it difficult to accurately measure minute displacements and angles.

[0004] Computer vision technology has been widely applied in civil engineering and shows promising potential in structural response measurement. This method typically involves fixing a target at the structural measurement point, capturing images with a camera, and analyzing the target's positional changes in the image sequence to calculate displacement. A key aspect of this method is its reliance on a static reference. Traditional approaches usually fix the camera on a stable, stationary base; however, when the target is far away, high-resolution cameras are required to ensure imaging accuracy, increasing costs. Using a drone platform to carry the camera offers high mobility and allows for close-range target imaging. However, the drone platform's own vibrations and attitude changes can infiltrate the image sequence, leading to decreased measurement accuracy. To address this issue, existing technologies use high-precision inertial measurement units for motion compensation or perform drone pose calculations based on video to eliminate the platform's motion influence, significantly increasing system complexity and computational burden. Therefore, achieving efficient and high-precision measurement of structural response requires further exploration. Summary of the Invention

[0005] To address the shortcomings of existing methods, this invention proposes a method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV).

[0006] This invention employs a cross-shaped laser emitting device to project a cross-shaped laser pattern onto a target as a static reference. It uses a drone to take close-up photos of the measurement area and achieves non-contact, high-precision displacement and rotation measurement based on the relative positional changes between the target and the projected pattern.

[0007] A method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) includes the following steps: A. Deploy a structural deformation measurement system based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area to be measured, and collect video data; B. Use YOLOv11 to identify targets in video images and extract regions of interest (ROIs); C. Through preliminary and precise two-stage localization, the exact positions of the visual target and the crosshair laser line are extracted; D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response.

[0008] Furthermore, step A specifically includes: A1. Design a dot array target fixed at the measurement point position. The target has at least 4 distinct dots for positioning, and the array layout has blank areas for clear imaging of the cross laser pattern. A2. At a stable fixed point near the measuring point, set up a cross laser emitting device and adjust its angle so that it projects a clear cross laser pattern on the target plane as a static spatial reference benchmark. A3. Control a drone equipped with a high-definition camera to fly to a position close to the target, hover and record a video sequence containing the visual target and the cross laser pattern as data for displacement and angle recognition; Furthermore, step B specifically includes: B1. Construct a dataset for training YOLOv11, which contains target images with cross laser projections taken at different angles, distances and lighting conditions; B2. Use the dataset to train and test YOLOv11, and use the trained model to identify and locate targets in video sequences, thereby extracting regions of interest.

[0009] Furthermore, step C specifically includes: C1. The initial positioning of the target and cross laser pattern adopts the multi-level Otsu threshold segmentation method, setting thresholds. T 1 and T 2. Divide the ROI region into three areas: target pattern, background, and cross laser pattern.

[0010] In the formula, R 1 represents the first category, the target pattern area; R 2 represents the second category, the blank area in the target; R 3 represents the third category, the cross-shaped laser pattern area; GrayGrayscale value; C2. Ellipse fitting is performed on the target pattern, and cross-line fitting is used on the cross laser pattern to achieve precise positioning of the target and the cross laser line; right R 1. Extract edge points from the region and fit the edge points into an ellipse, using the center of the ellipse as the coordinates of the positioning point; right R The point set in the 3 regions is fitted with intersecting lines to minimize the distance from the intersecting lines to the point set:

[0011] In the formula, X={ A 1, B 1, C 1, A 2, B 2, C 2} represents the parameters of the fitted line; { x i , y i}for R 3. Coordinates of the point set in the region; To solve for the parameter X, first initialize the equation of the line; then take the eigenvector corresponding to the largest eigenvalue of the covariance matrix of the point set. v 1 is the direction of the first straight line, and its orthogonal vector v 2 is used as the initial direction of another straight line, with the center point of the point set as the intersection point, to initialize the straight line. L 1,0 and L 2,0 ; Secondly, calculate the points from the two lines respectively. L 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion:

[0012] In the formula, P 1,j and P 1,j For the two sets of points; d 1,j From point set P to line L 1,j The distance; d 2,j From point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit straight lines respectivelyL 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided according to the minimum distance criterion and a straight line is fitted. The point set division and straight line fitting steps are repeated, and the iteration stops when the difference between the objective function of two adjacent iterations is less than the threshold.

[0013] Furthermore, step D specifically includes: D1. Establish a world coordinate system based on the target plane, calculate the homography matrix based on the pixel coordinates of the target and the world coordinates, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; D2. The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system.

[0014] In the formula, Δ d and Δ θ For the calculated displacement and rotation angle; X k and Y k for k The coordinates of the intersection point of the two straight lines in the frame; X 0 and Y 0 represents the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k for k The parameters of the first line in the frame; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

[0015] The present invention has the following advantages: (1) The method disclosed in this invention uses a drone as an observation platform, which can take close-up pictures of measurement points that are difficult to reach, and can measure high-altitude targets such as the bottom of bridges and the top of piers and towers. It has strong adaptability and non-contact advantages.

[0016] (2) The method disclosed in this invention uses static laser projection as an absolute reference benchmark and directly calculates the response based on the relative position change between the target and the laser pattern, eliminating the error caused by UAV vibration and avoiding complex UAV pose calculation, thus having high computational efficiency and measurement accuracy.

[0017] (3) The method disclosed in this invention uses a cross laser emission device for projection, and can simultaneously calculate the two-dimensional plane displacement and rotation of the target through the intersection point and slope change of the cross lines, so as to realize the synchronous acquisition of multiple response information. Attached Figure Description

[0018] Figure 1 This is a flowchart of the structural deformation measurement method of the present invention; Figure 2 This is a layout diagram of the test system of the present invention; Figure 3 This is the image ROI region extraction map of the present invention; Figure 4 This is the flow chart of the cross-line fitting algorithm of the present invention; Figure 5 This is a two-stage positioning diagram of the target and the cross laser line of the present invention; Figure 6 This is a schematic diagram illustrating the displacement and rotation calculation of the present invention; Figure 7 This is a diagram showing the measurement results of structural displacement and rotation angle of the present invention; Detailed Implementation

[0019] The following detailed description, with reference to the accompanying drawings, illustrates a method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV). The implementation process of this invention is as follows: Figure 1 As shown, the specific steps include: A. Deploy a structural deformation measurement system based on a static cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area, and acquire video data. The measurement system is as follows: Figure 2 As shown; A1. Using a continuous beam bridge as the test object, 2×2 point array targets were pasted at the bridge measuring points, with the target size selected as 50 mm×50 mm.

[0020] A2. At a stable fixed point near the measuring point, set up a 50 mW green crosshair laser emitting device, adjust the laser projection direction and distance, and project a clear and stable crosshair laser line onto the target plane as a static spatial reference benchmark.

[0021] A3. Using a DJI Mavic 3E drone, fly to a position directly facing the target, hover, and record a video sequence containing the visual target and the cross laser pattern. The drone camera resolution is 3840×2160, and the frame rate is 30 fps.

[0022] B. Use YOLOv11 to identify targets in video images and extract regions of interest (ROIs); B1. Take several images of targets under laser projection at different distances and angles, and expand the dataset using image enhancement techniques to serve as the YOLOv11 dataset.

[0023] B2. The dataset was used for YOLOv11 model training and testing in an 8:2 ratio. The trained model was then used to identify targets in video images as Regions of Interest (ROIs). Some of the recognition results are shown in the image below. Figure 3 As shown.

[0024] C. Through preliminary and precise two-stage localization, the exact positions of the visual target and the crosshair laser line are extracted; C1. A multi-level Otsu thresholding method is used to divide the ROI region into three areas: the target pattern, the background, and the cross laser pattern, achieving initial localization of the target and the cross laser pattern; the RGB image is converted to a grayscale image, and a threshold is set. T 1 and T 2. Divide the image into 3 categories:

[0025] In the formula, R 1 represents the first category, the target pattern area; R 2 represents the second category, the blank area in the target; R 3 represents the third category, the cross-shaped laser pattern area; Gray This is the grayscale value.

[0026] C2. Ellipse fitting is applied to the target pattern, and cross-line fitting is applied to the cross laser pattern to achieve precise positioning of the target and the cross laser line; extraction R 1. Substitute the edge points of the region into the following matrix to solve for the parameters of the general equation of the ellipse:

[0027] In the formula, { a , b , c , d , e} represents the parameters of the general equation of an ellipse; the coordinates of the center of the ellipse can be calculated as follows:

[0028] In the formula,x 0 and y 0 represents the coordinates of the center of the ellipse.

[0029] right R The specific steps for fitting cross lines to the point set in the 3 regions are as follows: Figure 4 As shown; the objective is to minimize the distance from the intersecting lines to the set of points:

[0030] In the formula, X={ A 1, B 1, C 1, A 2, B 2, C 2} represents the parameters of the fitted line; { x i , y i}for R 3. Point set coordinates of the region; To solve for the parameter X, we first initialize the equation of the line, and then take the eigenvector corresponding to the largest eigenvalue of the covariance matrix of the point set. v 1 is the direction of the first straight line, and its orthogonal vector v 2, as the initial direction of another straight line, with the center of the point set as the intersection point, the initialized straight line can be represented as: L 1,0 and L 2,0 :

[0031] In the formula, v 1 represents the eigenvector corresponding to the largest eigenvalue of the covariance matrix; v 2 is with v 1. Orthogonal vectors; calculate the distances from the point set P to the line. L 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion:

[0032] In the formula, P 1,j and P 1,j For the two sets of points; d 1,j From point set P to line L 1,j The distance; d 2,j From point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit straight lines respectively L 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided and a line is fitted. The point set division and line fitting steps are repeated. Iteration stops when the difference between the objective functions of two adjacent iterations is less than a threshold. The cutoff condition is:

[0033] In the formula, ε The cutoff threshold is set to 1×10 in this example. -6 Preliminary and precise two-stage target and cross-laser positioning results are as follows: Figure 5 As shown.

[0034] D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response.

[0035] D1. Calculate the homography matrix based on the known physical dimensions of the visual target, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; establish a world coordinate system with the target plane as the center, and Q1Q2 as the coordinate system. x The axis, and Q1Q4 orthogonal to it are y Axis. Calculate the homography matrix based on the pixel coordinates of targets Q1~Q4 and the world coordinates:

[0036] In the formula, x p and y p These are the coordinates of the feature point in the pixel coordinate system; x w and y w H represents the coordinates of the feature point in the world coordinate system; H is the homography transformation matrix. h ij The elements are the homography transformation matrix; substitute the coordinates of the four point pairs in the target to calculate the homography matrix.

[0037] The equations of the two straight lines at pixel coordinates can be represented in matrix form:

[0038] Through homography matrix transformation, the equations of the two lines in the world coordinate system are:

[0039] D2. Solve for displacement and rotation based on the intersection points and slope changes of the intersecting lines in each frame, such as... Figure 6 As shown; The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system.

[0040] In the formula, Δ d and Δ θ For the calculated displacement and rotation angle; X k and Y k for k The coordinates of the intersection point of the two straight lines in the frame; X 0 and Y 0 represents the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k for k The parameters of the first line in the frame; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

Claims

1. A method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV), characterized in that, Includes the following steps: A. Deploy a structural deformation measurement system based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area to be measured, and collect video data; B. Use the YOLOv11 model to identify targets in video images and extract the Region of Interest (ROI). C. Through preliminary and precise two-stage localization, the accurate positions of the visual target and the crosshair laser line are extracted; specifically including: C1. The initial localization of the target and cross laser pattern uses a multi-level Otsu threshold segmentation method, setting thresholds T1 and T2 to divide the ROI region into three areas: the target pattern, the background, and the cross laser pattern. （1） In the formula, R1 is the first type, the target pattern area; R2 is the second type, the blank area in the target; and R3 is the third type, the cross laser pattern area. C2. The precise positioning of the target is achieved by extracting the sub-pixel coordinates of the center of the dot using an ellipse fitting method; the precise positioning of the cross laser pattern is achieved by extracting the sub-pixel coordinates of the cross laser using an iterative fitting algorithm that minimizes the distance from the point to the line. The cross-line iterative fitting algorithm aims to minimize the distance from the cross-line to the point set: （2） In the formula, X={A1, B1, C1, A2, B2, C2} are the parameters of the fitted line; {x i , y i } represents the coordinates of the point set in region R3; To solve for the parameter X, first initialize the equation of the line; take the eigenvector v1 corresponding to the largest eigenvalue of the covariance matrix of the point set as the direction of the first line, and its orthogonal vector v2 as the initial direction of the other line. Use the center point of the point set as the intersection point to initialize the line L. 1,0 and L 2,0 ; Secondly, calculate the points from the set to the two lines L. 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion: （3） In the formula, P 1,j and P 2,j For the two sets of points to be divided; d 1,j For point set P to line L 1,j distance; d 2,j For point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit the straight line L respectively 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided according to the minimum distance criterion and a straight line is fitted. The point set division and straight line fitting steps are repeated, and the iteration stops when the difference between the objective function of two adjacent iterations is less than the threshold. D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response; specifically including: D1. Establish a world coordinate system based on the target plane, calculate the homography matrix based on the pixel coordinates of the target and the world coordinates, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; D2. The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system. （4） In the formula, Δd and Δθ are the calculated displacement and rotation angle; X k and Y k Let X0 and Y0 be the coordinates of the intersection of the two lines in frame k; X0 and Y0 are the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k The parameter for the first straight line in frame k; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

2. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step A specifically includes: A1. Design a dot array target fixed at the measurement point position. The target has at least 4 distinct dots for positioning, and the array layout has blank areas for clear imaging of the cross laser pattern. A2. At a stable fixed point near the measuring point, set up a cross laser emitting device and adjust its angle so that it projects a clear cross laser pattern on the target plane as a static spatial reference benchmark. A3. Control a drone equipped with a high-definition camera to fly to a position close to the target, hover and record a video sequence containing the visual target and the cross laser pattern as data for displacement and angle recognition.

3. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step B specifically includes: B1. Construct a dataset for training the YOLOv11 model, the dataset containing target images with cross laser projections taken at different angles, distances and lighting conditions; B2. Use the dataset to train and test the YOLOv11 model, and use the trained model to identify and locate targets in the video sequence, thereby extracting the region of interest.

Images