Structural deformation measurement method based on cross laser reference and unmanned aerial vehicle

Abstract

A structural deformation measurement method based on a cross laser reference and an unmanned aerial vehicle comprises the steps that a cross laser emitting device is adopted, and a cross laser pattern is projected on a target to serve as a static reference; an unmanned aerial vehicle is adopted to shoot a measuring point area in a close range, non-contact and high-precision displacement and rotation angle measurement is achieved according to the relative position change of a target and a projection pattern, and the method comprises the steps that A, a cross laser reference and unmanned aerial vehicle-based structural deformation measurement system is arranged in a target area to be measured, and video data acquisition is performed; b, identifying a target in the video image by adopting YOLOv11, and extracting a region of interest (ROI); c, extracting accurate positions of the visual target and the cross laser line through preliminary and accurate two-stage positioning; d, analyzing the relative position change of the target and the cross laser ray, and calculating the displacement and rotation angle response of the target; according to the measuring method, the static laser reference is introduced, and the relative change of the target and the laser pattern is analyzed, so that the synchronous measurement of the two-dimensional displacement and the rotating angle is realized, and the measuring method has the advantages of non-contact and high-precision measurement.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of structural deformation measurement method based on cross laser reference and unmanned aerial vehicle, belong to structural detection field. BACKGROUND

[0002] Structures (bridges, buildings, etc.) are subjected to traffic loads, wind loads, temperature changes and other factors during service, and the displacement or angular response of the key parts is an important parameter for evaluating the health status and safety performance of the structure. Therefore, it is of great significance to develop efficient and accurate structural response measurement methods to ensure the safe service of the structure.

[0003] Existing structural response measurement can be divided into contact and non-contact methods. Contact methods mainly use LVDT displacement sensors, dial gauges, inclinometers and other equipment. Contact measurement methods usually require sensors to be installed at measurement points. For areas that are difficult for personnel to access (such as the bottom of a bridge or the top of a high pier), there are difficulties in implementation and high costs. Non-contact methods mainly include total station and global positioning system (GPS). Total station has high static measurement accuracy, but it is difficult to achieve dynamic continuous measurement of the target. GPS technology can achieve remote monitoring, but it lacks stability in environments where satellite signals are blocked, making it difficult to accurately measure small displacements and angles.

[0004] Computer vision technology has been widely used in civil engineering and has good prospects in structural response measurement. This method usually fixes targets at structural measurement points and analyzes the position changes of the targets in image sequences to calculate displacement. The key to this method is that it needs a static reference. Traditional solutions usually fix the camera on a stable and stationary base, but when the target is far away, high-resolution camera equipment is needed to ensure imaging accuracy, resulting in increased cost. Using a UAV platform to carry a camera has high mobility and can take close-up shots of the target. The vibration and attitude changes of the UAV platform itself can interfere with the image sequence, resulting in a decrease in measurement accuracy. To solve this problem, existing technologies use high-precision inertial measurement units for motion compensation or use video to calculate the UAV pose to eliminate the effects of platform motion, significantly increasing the complexity and computational burden of the system. Therefore, further exploration is needed to achieve efficient and accurate structural response measurement. SUMMARY

[0005] In view of the shortcomings of existing methods, the present application proposes a structural deformation measurement method based on cross laser reference and unmanned aerial vehicle.

[0006] The present application uses a cross laser emitting device to project a cross laser pattern on the target as a static reference. A UAV is used to take close-up shots of the measurement point area, and based on the relative position changes of the target and the projected pattern, non-contact and high-precision displacement and angle measurement is achieved.

[0007] A method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) includes the following steps: A. Deploy a structural deformation measurement system based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area to be measured, and collect video data; B. Use YOLOv11 to identify targets in video images and extract regions of interest (ROIs); C. Through preliminary and precise two-stage localization, the exact positions of the visual target and the crosshair laser line are extracted; D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response.

[0008] Furthermore, step A specifically includes: A1. Design a dot array target fixed at the measurement point position. The target has at least 4 distinct dots for positioning, and the array layout has blank areas for clear imaging of the cross laser pattern. A2. At a stable fixed point near the measuring point, set up a cross laser emitting device and adjust its angle so that it projects a clear cross laser pattern on the target plane as a static spatial reference benchmark. A3. Control a drone equipped with a high-definition camera to fly to a position close to the target, hover and record a video sequence containing the visual target and the cross laser pattern as data for displacement and angle recognition; Furthermore, step B specifically includes: B1. Construct a dataset for training YOLOv11, which contains target images with cross laser projections taken at different angles, distances and lighting conditions; B2. Use the dataset to train and test YOLOv11, and use the trained model to identify and locate targets in video sequences, thereby extracting regions of interest.

[0009] Furthermore, step C specifically includes: C1. The initial positioning of the target and cross laser pattern adopts the multi-level Otsu threshold segmentation method, setting thresholds. T 1 and T 2. Divide the ROI region into three areas: target pattern, background, and cross laser pattern.

[0010] In the formula, R 1 represents the first category, the target pattern area; R 2 represents the second category, the blank area in the target; R 3 represents the third category, the cross-shaped laser pattern area; GrayGrayscale value; C2. Ellipse fitting is performed on the target pattern, and cross-line fitting is performed on the cross laser pattern to achieve precise positioning of the target and the cross laser line; right R 1. Extract edge points from the region and fit the edge points into an ellipse, using the center of the ellipse as the coordinates of the positioning point; right R The point set in the 3 regions is fitted with intersecting lines to minimize the distance from the intersecting lines to the point set:

[0011] In the formula, X={ A 1, B 1, C 1, A 2, B 2, C 2} represents the parameters of the fitted line; { x i , y i}for R 3. Coordinates of the point set in the region; To solve for the parameter X, first initialize the equation of the line; then take the eigenvector corresponding to the largest eigenvalue of the covariance matrix of the point set. v 1 is the direction of the first straight line, and its orthogonal vector v 2 is used as the initial direction of another straight line, with the center point of the point set as the intersection point, to initialize the straight line. L 1,0 and L 2,0 ; Secondly, calculate the points from the two lines respectively. L 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion:

[0012] In the formula, P 1,j and P 1,j For the two sets of points; d 1,j From point set P to line L 1,j The distance; d 2,j From point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit straight lines respectivelyL 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided according to the minimum distance criterion and a straight line is fitted. The point set division and straight line fitting steps are repeated, and the iteration stops when the difference between the objective function of two adjacent iterations is less than the threshold.

[0013] Furthermore, step D specifically includes: D1. Establish a world coordinate system based on the target plane, calculate the homography matrix based on the pixel coordinates of the target and the world coordinates, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; D2. The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system.

[0014] In the formula, Δ d and Δ θ For the calculated displacement and rotation angle; X k and Y k for k The coordinates of the intersection point of the two straight lines in the frame; X 0 and Y 0 represents the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k for k The parameters of the first line in the frame; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

[0015] The present invention has the following advantages: (1) The method disclosed in this invention uses a drone as an observation platform, which can take close-up pictures of measurement points that are difficult to reach, and can measure high-altitude targets such as the bottom of bridges and the top of piers and towers. It has strong adaptability and non-contact advantages.

[0016] (2) The method disclosed in this invention uses static laser projection as an absolute reference benchmark and directly calculates the response based on the relative position change between the target and the laser pattern, eliminating the error caused by UAV vibration and avoiding complex UAV pose calculation, thus having high computational efficiency and measurement accuracy.

[0017] (3) The method disclosed in this invention uses a cross laser emission device for projection, and can simultaneously calculate the two-dimensional plane displacement and rotation of the target through the intersection point and slope change of the cross lines, so as to realize the synchronous acquisition of multiple response information. Attached Figure Description

[0018] Figure 1 This is a flowchart of the structural deformation measurement method of the present invention; Figure 2 This is a layout diagram of the test system of the present invention; Figure 3 This is the image ROI region extraction map of the present invention; Figure 4 This is the flow chart of the cross-line fitting algorithm of the present invention; Figure 5 This is a two-stage positioning diagram of the target and the cross laser line of the present invention; Figure 6 This is a schematic diagram illustrating the displacement and rotation calculation of the present invention; Figure 7 This is a diagram showing the measurement results of structural displacement and rotation angle of the present invention; Detailed Implementation

[0019] The following detailed description, with reference to the accompanying drawings, illustrates a method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV). The implementation process of this invention is as follows: Figure 1 As shown, the specific steps include: A. Deploy a structural deformation measurement system based on a static cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area, and acquire video data. The measurement system is as follows: Figure 2 As shown; A1. Using a continuous beam bridge as the test object, 2×2 point array targets were pasted at the bridge measuring points, with the target size selected as 50 mm×50 mm.

[0020] A2. At a stable fixed point near the measuring point, set up a 50 mW green crosshair laser emitting device, adjust the laser projection direction and distance, and project a clear and stable crosshair laser line onto the target plane as a static spatial reference benchmark.

[0021] A3. Using a DJI Mavic 3E drone, fly to a position directly facing the target, hover, and record a video sequence containing the visual target and the cross laser pattern. The drone camera resolution is 3840×2160, and the frame rate is 30 fps.

[0022] B. Use YOLOv11 to identify targets in video images and extract regions of interest (ROIs); B1. Take several images of targets under laser projection at different distances and angles, and expand the dataset using image enhancement techniques to serve as the YOLOv11 dataset.

[0023] B2. The dataset was used for YOLOv11 model training and testing in an 8:2 ratio. The trained model was then used to identify targets in video images as Regions of Interest (ROIs). Some of the recognition results are shown in the image below. Figure 3 As shown.

[0024] C. Through preliminary and precise two-stage localization, the exact positions of the visual target and the crosshair laser line are extracted; C1. A multi-level Otsu thresholding method is used to divide the ROI region into three areas: the target pattern, the background, and the cross laser pattern, achieving initial localization of the target and the cross laser pattern; the RGB image is converted to a grayscale image, and a threshold is set. T 1 and T 2. Divide the image into 3 categories:

[0025] In the formula, R 1 represents the first category, the target pattern area; R 2 represents the second category, the blank area in the target; R 3 represents the third category, the cross-shaped laser pattern area; Gray This is the grayscale value.

[0026] C2. Ellipse fitting is applied to the target pattern, and cross-line fitting is used to fit the cross laser pattern, achieving precise positioning of the target and the cross laser line; extraction R 1. Substitute the edge points of the region into the following matrix to solve for the parameters of the general equation of the ellipse:

[0027] In the formula, { a , b , c , d , e} represents the parameters of the general equation of an ellipse; the coordinates of the center of the ellipse can be calculated as follows:

[0028] In the formula,x 0 and y 0 represents the coordinates of the center of the ellipse.

[0029] right R The specific steps for fitting cross lines to the point set in the 3 regions are as follows: Figure 4 As shown; the objective is to minimize the distance from the intersecting lines to the set of points:

[0030] In the formula, X={ A 1, B 1, C 1, A 2, B 2, C 2} represents the parameters of the fitted line; { x i , y i}for R 3. Point set coordinates of the region; To solve for the parameter X, we first initialize the equation of the line, and then take the eigenvector corresponding to the largest eigenvalue of the covariance matrix of the point set. v 1 is the direction of the first straight line, and its orthogonal vector v 2, as the initial direction of another straight line, with the center of the point set as the intersection point, the initialized straight line can be represented as: L 1,0 and L 2,0 :

[0031] In the formula, v 1 represents the eigenvector corresponding to the largest eigenvalue of the covariance matrix; v 2 is with v 1. Orthogonal vectors; calculate the distances from the point set P to the line. L 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion:

[0032] In the formula, P 1,j and P 1,j For the two sets of points; d 1,j From point set P to line L 1,j The distance; d 2,j From point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit straight lines respectively L 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided and a line is fitted. The point set division and line fitting steps are repeated. Iteration stops when the difference between the objective functions of two adjacent iterations is less than a threshold. The cutoff condition is:

[0033] In the formula, ε The cutoff threshold is set to 1×10 in this example. -6 Preliminary and precise two-stage target and cross-laser positioning results are as follows: Figure 5 As shown.

[0034] D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response.

[0035] D1. Calculate the homography matrix based on the known physical dimensions of the visual target, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; establish a world coordinate system with the target plane as the center, and Q1Q2 as the coordinate system. x The axis, and Q1Q4 orthogonal to it are y Axis. Calculate the homography matrix based on the pixel coordinates of targets Q1~Q4 and the world coordinates:

[0036] In the formula, x p and y p These are the coordinates of the feature point in the pixel coordinate system; x w and y w H represents the coordinates of the feature point in the world coordinate system; H is the homography transformation matrix. h ij The elements are the homography transformation matrix; substitute the coordinates of the four point pairs in the target to calculate the homography matrix.

[0037] The equations of the two straight lines at pixel coordinates can be represented in matrix form:

[0038] Through homography matrix transformation, the equations of the two lines in the world coordinate system are:

[0039] D2. Solve for displacement and rotation based on the intersection points and slope changes of the intersecting lines in each frame, such as... Figure 6 As shown; The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system.

[0040] In the formula, Δ d and Δ θ For the calculated displacement and rotation angle; X k and Y k for k The coordinates of the intersection point of the two straight lines in the frame; X 0 and Y 0 represents the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k for k The parameters of the first line in the frame; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

Claims

1. A method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV), characterized in that, Includes the following steps: A. Deploy a structural deformation measurement system based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) in the target area to be measured, and collect video data; B. Use the YOLOv11 model to identify targets in video images and extract the Region of Interest (ROI). C. Through preliminary and precise two-stage localization, the exact positions of the visual target and the crosshair laser line are extracted; D. Analyze the relative positional changes of the target and the crosshair laser line, and calculate the target's displacement and rotational response.

2. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step A specifically includes: A1. Design a dot array target fixed at the measurement point position. The target has at least 4 distinct dots for positioning, and the array layout has blank areas for clear imaging of the cross laser pattern. A2. At a stable fixed point near the measuring point, set up a cross laser emitting device and adjust its angle so that it projects a clear cross laser pattern on the target plane as a static spatial reference benchmark. A3. Control a drone equipped with a high-definition camera to fly to a position close to the target, hover and record a video sequence containing the visual target and the cross laser pattern as data for displacement and angle recognition.

3. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step B specifically includes: B1. Construct a dataset for training the YOLOv11 model, the dataset containing target images with cross laser projections taken at different angles, distances and lighting conditions; B2. Use the dataset to train and test the YOLOv11 model, and use the trained model to identify and locate targets in the video sequence, thereby extracting the region of interest.

4. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step C specifically includes: C1. The initial positioning of the target and cross laser pattern adopts the multi-level Otsu threshold segmentation method, setting thresholds. T 1 and T 2. Divide the ROI region into three areas: target pattern, background, and cross laser pattern. In the formula, R 1 represents the first category, the target pattern area; R 2 represents the second category, the blank area in the target; R 3 represents the third category, the cross-shaped laser pattern area; C2. The precise positioning of the target is achieved by extracting the sub-pixel coordinates of the center of the dot using an ellipse fitting method; the precise positioning of the cross laser pattern is achieved by extracting the sub-pixel coordinates of the cross laser using an iterative fitting algorithm that minimizes the distance from the point to the line. The cross-line iterative fitting algorithm aims to minimize the distance from the cross-line to the point set: In the formula, X={ A 1, B 1, C 1, A 2, B 2, C 2} represents the parameters of the fitted line; { x i , y i }for R 3. Point set coordinates of the region; To solve for the parameter X, first initialize the equation of the line; then take the eigenvector corresponding to the largest eigenvalue of the covariance matrix of the point set. v 1 is the direction of the first straight line, and its orthogonal vector v 2 is used as the initial direction of another straight line, with the center point of the point set as the intersection point, to initialize the straight line. L 1,0 and L 2,0 ; Secondly, calculate the points from the two lines respectively. L 1,j and L 2,j The distance is used to partition the point set according to the minimum distance criterion: In the formula, P 1,j and P 1,j For the two sets of points; d 1,j From point set P to line L 1,j The distance; d 2,j From point set P to line L 2,j The distance; p i The coordinates of the total point set; Furthermore, according to P 1,j and P 2,j Fit straight lines respectively L 1,j+1 and L 2,j+1 Update parameter X j+1 ; Recalculate the point set to the new line L 1,j+1 and L 2,j+1 The distance is calculated, and the point set is divided according to the minimum distance criterion and a straight line is fitted. The point set division and straight line fitting steps are repeated, and the iteration stops when the difference between the objective function of two adjacent iterations is less than the threshold.

5. The method for measuring structural deformation based on a cross-shaped laser reference and an unmanned aerial vehicle (UAV) according to claim 1, characterized in that, Step D specifically includes: D1. Establish a world coordinate system based on the target plane, calculate the homography matrix based on the pixel coordinates of the target and the world coordinates, and establish the mapping relationship between the image pixel coordinate system and the world physical coordinate system; D2. The structural displacement response is obtained by calculating the coordinate change of the intersection point of the cross laser pattern in the world coordinate system; the rotation response is obtained by calculating the slope change of the center line of the cross laser pattern in the world coordinate system. In the formula, Δ d and Δ θ For the calculated displacement and rotation angle; X k and Y k for k The coordinates of the intersection point of the two straight lines in the frame; X 0 and Y 0 represents the coordinates of the intersection of the two lines in the initial frame; A 1w,k and B 1w,k for k The parameters of the first line in the frame; A 1w,0 and B 1w,0 The parameters for the first straight line in the initial frame.

Images