Displacement measurement method based on UAV images and digital images

Abstract

A displacement measurement method based on drone images and digital images includes the following steps: Step 1: 11) fixing the camera position; 12) temporarily fixing the calibration plate on the target plane and shooting to obtain a first image; 13) fixing the calibration plate near the target plane and shooting to obtain a second image; 14) making the X in the world coordinate system W O W Y W The plane is coplanar with the target plane to obtain a standard world coordinate system; solve the coordinates of all calibration plates in the standard world coordinate system; Step 2: 21) Drive the drone to hover at a suitable position and use the camera to shoot video; 22) Convert the first image into a standard reference image, and calculate the homography matrix of the standard reference image and each frame image relative to the target plane; 23) Calculate the homography matrix of each frame image in the video relative to the standard reference image; 24) Use the digital image correlation method to solve the pixel displacement of the target point, and use the size factor to convert the pixel displacement into actual physical displacement.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and visual measurement technology, and specifically provides a displacement measurement method based on drone images and digital images. Background Art

[0002] Displacement is one of the most important parameters for assessing the integrity of civil infrastructure, such as buildings, bridges, and tunnels. Traditional displacement measurement technologies include differential transformer displacement sensors, laser displacement sensors, and the Global Navigation Satellite System (GNSS). However, these various sensor deployments suffer from high cost, complex operation, and distributed measurement points, making them difficult to meet the requirements for displacement measurement of large structures. While GNSS can provide non-contact wireless measurement, its measurement locations are limited and costly.

[0003] As a non-contact displacement measurement method, the traditional Digital Image Correlation (DIC) method offers advantages such as simplicity, low cost, and full-field measurement. It has gained significant attention and development in fields such as materials engineering, civil engineering, and machinery, and has now entered the commercial stage. However, the traditional DIC method requires the camera to be stationary during measurement, otherwise errors will occur, which greatly limits its application. For example, when measuring large-span, tall structures such as bridges and wind turbines, suitable on-site camera mounting platforms are unavailable.

[0004] As a flying platform, drones offer advantages such as high maneuverability, flexibility, and mature applications. Displacement measurement methods based on drone images offer new approaches for measuring the displacement of large-span, tall structures. However, even when hovering, drones can slip and wobble due to environmental influences such as air flow, which can cause the final measurement results to include camera motion information. Eliminating the effects of drone motion on displacement measurements has become a hot topic of research both domestically and internationally. Commonly used methods include: 1) using a high-pass filter to remove drone motion from the measurement results; 2) calculating and eliminating drone motion using information recorded by an inertial measurement unit (IMU); and 3) using a fixed background calibration plate to eliminate the effects of drone motion. These methods can mitigate the effects of drone motion on measurements to a certain extent in specific situations, but they also have certain drawbacks. The first method assumes that the drone's vibration frequency is significantly lower than the fundamental frequency of the target and requires sufficient measurement time, which makes it ineffective for measuring low-frequency structures or short-duration measurements. The second method requires that the camera and drone are absolutely fixed and requires a longer measurement period. The third method requires that the fixed background calibration plate be coplanar with the measurement plane, or parallel to the measured plane and the distance between them is known, otherwise the method will fail. This is difficult to meet in actual measurement work due to environmental conditions. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a displacement measurement method based on drone images and digital images. It only requires fixing a calibration plate arbitrarily near the target plane. On the basis of introducing the digital image correlation method, drone photography is used to realize non-contact full-field displacement measurement of the measurement target.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A displacement measurement method based on drone images and digital images includes the following steps:

[0008] Step 1: Solve the coordinates of the calibration plate in the world coordinate system

[0009] 11) Fix the camera position and adjust the camera angle so that the camera can capture the target plane and calibration plate;

[0010] 12) temporarily fixing the calibration plate on the target plane, and using a fixed camera to capture an image including the target plane and all calibration plates to obtain a first image;

[0011] 13) Fixing the calibration plate near the target plane, and using a fixed camera to capture an image including the target plane and all calibration plates to obtain a second image;

[0012] 14) Make X in the world coordinate system W O W Y W The plane is coplanar with the target plane, and a standard world coordinate system is obtained; the coordinates of all calibration plates in the second image in the standard world coordinate system are solved;

[0013] Step 2: Measure displacement

[0014] 21) Drive the drone carrying the camera to hover at a suitable position, and use the camera to shoot the area including the target plane and all calibration plates to obtain a video;

[0015] 22) converting the first image into a standard reference image, and calculating the homography matrix of the standard reference image and each frame image relative to the target plane according to the coordinates and pixel coordinates of the calibration plate in the standard world coordinate system;

[0016] 23) Calculate the homography matrix of each frame image in the video relative to the standard reference image;

[0017] 24) Use digital image correlation method to solve the pixel displacement of the target point, and use the size factor to convert the pixel displacement into actual physical displacement.

[0018] Furthermore, in step 14), the coordinates of the calibration plate in the second image in the standard world coordinate system are solved as follows:

[0019] When the calibration plate is temporarily fixed, the homogeneous coordinate transformation relationship between the world coordinates and the camera coordinates of the calibration plate is:

[0020]

[0021] Among them, (x w0 ,y w0 ,z w0 ) represents the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane; M0 represents the external parameter matrix of the camera in the standard world coordinate system; (x c0 ,y c0 ,z c0 ) represents the coordinates of the calibration plate in the camera coordinate system when it is temporarily fixed on the target plane; r ij Parameters representing the camera's posture in the standard world coordinate system; t i Represents the position parameters of the camera in the standard world coordinate system; i = 1, 2, 3, j = 1, 2, 3;

[0022] After the calibration plate is fixed, an auxiliary world coordinate system is created to move with the calibration plate, so that the coordinates of the calibration plate in the auxiliary world coordinate system are the same as the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane. The homogeneous coordinate transformation relationship between the coordinates of the calibration plate in the auxiliary world coordinate system and the camera coordinates is:

[0023]

[0024] Where M' represents the external parameter matrix of the camera in the auxiliary world coordinate system; (x c1 ,y c1 ,z c1 ) represents the coordinates of the calibration plate in the camera coordinate system after it is fixed; r′ ij Parameters representing the camera's posture in the auxiliary world coordinate system; t' i Represents the position parameters of the camera in the auxiliary world coordinate system; i = 1, 2, 3, j = 1, 2, 3;

[0025] Since the camera is fixed during the process of capturing the first and second images, we have:

[0026]

[0027] The coordinates of the calibration plate in the standard world coordinate system after being fixed are obtained:

[0028]

[0029] Among them, (x w1 ,y w1 ,zw1 ) represents the coordinates of the calibration plate in the standard world coordinate system after it is fixed.

[0030] Furthermore, in step 22), the method of converting the first image into a standard reference image using the homography matrix is:

[0031] Let the size factor be:

[0032]

[0033] Among them, f u and f v Represent the size factors in the U direction and V direction in the pixel coordinate system respectively; l1 represents the distance between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; l2 represents the distance between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed; (Δu1, Δv1) represents the pixel coordinate difference between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; (Δu2, Δv2) represents the pixel coordinate difference between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed;

[0034] Define a reference point on the first image so that the pixel coordinates (u0, v0) of the reference point in the standard reference image are the same as those in the first image. Calculate the pixel coordinates of any point in the standard reference image as follows:

[0035]

[0036] Among them, (u i ,v i ) represents the pixel coordinates of any point in the standard reference image, (Δx i ,Δy i ) represents any point (u i ,v i ) and the reference point (u0, v0) in the standard world coordinate system; and:

[0037]

[0038] Where (u, v) represents the pixel coordinates of a point in the standard reference image; (u1, v1) represents the pixel coordinates of the point in the first image corresponding to the point (u, v) in the standard reference image; H′ represents the homography matrix between the first image and the standard reference image; h ij Parameters representing image deformation characteristics, i = 1, 2, 3, j = 1, 2, 3;

[0039] Since the homography matrix H′ has 8 unknowns, the pixel coordinates of the corresponding 4 pairs of points in the first image and the standard reference image are used to solve the unknowns in the homography matrix H′, and the homography matrix H′ between the first image and the standard reference image is obtained. The homography matrix H′ is used to convert the first image into the standard reference image.

[0040] The beneficial effects of the present invention are:

[0041] The present invention is based on a displacement measurement method using drone images and digital images. After solving the coordinates of the calibration plate in the world coordinate system, the drone is used to shoot video to measure the displacement. Specifically, when solving the coordinates of the calibration plate in the world coordinate system, the camera is first fixed in a suitable position so that the camera's shooting range includes the target plane and all calibration plates; then all calibration plates are temporarily fixed on the target plane for shooting, and then the calibration plates are fixed near the target plane for shooting, to obtain the first image and the second image respectively; finally, the X coordinate of the world coordinate system is calculated. W O W Y W The plane is coplanar with the target plane to obtain a standard world coordinate system. Taking advantage of the fact that the camera's position remains fixed during the capture of the first and second images, the coordinates of the calibration plate in the first image in the standard world coordinate system are used to determine the coordinates of the corresponding calibration plate in the second image in the standard world coordinate system. When measuring displacement, the camera is first carried by a drone and hovered in a suitable position so that the camera's camera range includes the target plane and the area where all calibration plates are located. The first image is then converted into a standard reference image. The coordinates of the calibration plate in the standard world coordinate system and the pixel coordinates are used to calculate the quasi-reference image and the homography matrix of each frame relative to the target plane, thereby obtaining the homography matrix of each frame in the video relative to the standard reference image. Each frame in the video is then transformed to the plane where the standard reference image is located, thereby eliminating the error caused by the shaking of the drone. Finally, the pixel displacement of the target point is solved using the digital image correlation method, and the pixel displacement is converted into actual physical displacement using the size factor.

[0042] In summary, in the displacement measurement method based on drone images and digital images of the present invention, it is only necessary to fix the calibration plate arbitrarily near the target plane, that is, the calibration plate can be fixed at a convenient fixed position during the measurement process, and there is no strict restriction on the fixed position of the calibration plate, which greatly improves the feasibility of measurement and makes the measurement operation more convenient; in addition, by introducing a standard reference image, all images are calculated under the same size factor, which simplifies the size calibration process after calculation by the digital image correlation method, eliminates the error caused by the shaking of the drone, and realizes non-contact full-field displacement measurement of the measurement target. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to make the purpose, technical solutions and beneficial effects of the present invention more clear, the present invention provides the following drawings for illustration:

[0044] Figure 1 This is the camera imaging principle diagram;

[0045] Figure 2 It is a mapping relationship diagram between two planes;

[0046] Figure 3 The transformation principle diagram of images taken at different viewing angles;

[0047] Figure 4 Schematic diagram of the calibration plate after temporary fixation and fixation. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention.

[0049] The Digital Image Correlation (DIC) method uses a camera to obtain digital images of the measured surface before and after deformation. The displacement of each point on the measured surface is then determined by matching corresponding image subsets within the digital images. Because the displacements obtained by the DIC method are based on digital images, the result is pixel displacement. To convert this result into actual displacement in physical units, the camera must be calibrated. The calibration method is as follows.

[0050] The camera imaging model generally adopts the pinhole imaging principle. To facilitate the explanation of the camera imaging principle, the world coordinate system O is established. W X W Y W Z W , camera coordinate system O C X C Y C Z C , image coordinate system OXY, pixel coordinate system O p UV, such as Figure 1 shown.

[0051] According to the pinhole imaging principle, the mapping relationship between the pixel coordinate system and the world coordinate system is:

[0052]

[0053] Among them, (x w ,y w ,z w ) represents the coordinates of the world coordinate system; (x c ,yc ,z c ) represents the coordinates of the camera coordinate system; (u, v) represents the coordinates of the pixel coordinate system; M1 represents the camera internal parameter matrix; f u and f v Represent the size factors in the U and V directions in the pixel coordinate system respectively; (u0, v0) represents the coordinates of the camera principal point in the pixel coordinate system; M2 represents the camera's external parameter matrix, which is used to describe the camera's position in the world coordinate system, where R and T are the camera's rotation matrix and position vector, respectively, and their expressions are:

[0054]

[0055] Among them, r ij Represents the camera's posture parameters in the world coordinate system, t i Represents the position parameters in the camera world coordinate system, i = 1, 2, 3, j = 1, 2, 3;

[0056] The camera's intrinsic parameter matrix M1 is only related to the camera itself and can be determined through a single camera calibration. The camera's extrinsic parameter matrix M2 changes with camera rotation and translation. Traditional DIC methods require that the camera be fixed during measurement and the optical axis be perpendicular to the target plane. In this case, the world coordinate system and the camera coordinate system can be considered to completely coincide. The camera's extrinsic parameter matrix is ​​then:

[0057]

[0058] Then simplify formula (1) to:

[0059]

[0060] Then the actual displacement of the measurement target can be expressed as:

[0061]

[0062] However, when the camera is connected to a drone, the camera will shake along with the movement of the drone, and the camera's external parameter matrix M2 will change, so the above method will no longer be applicable.

[0063] The influence of drone shaking can be effectively eliminated by using a fixed background calibration plate. The specific operation is to solve the homography matrix H between the measured plane and the camera imaging plane, transform all images into images taken at the same perspective, and then use the DIC algorithm to solve the displacement of the measurement target. Therefore, the key lies in solving the homography matrix. The homography matrix is ​​used to represent the mapping relationship between two planes in projective geometry. Its physical meaning can be understood as the projection matrix of a plane on another plane through the projection center. Its physical meaning is as follows Figure 2shown.

[0064] From its physical meaning, we know that the projection between two planes is mutual, so the homography matrix is ​​a reversible matrix. Figure 2 In the equation, the relationship between the two planes is:

[0065]

[0066] Among them, (x, y) represents the coordinates of the point on the original plane, (x1, y1) represents the coordinates of the corresponding point on plane 1, and H1 represents the mapping relationship between plane 1 and the original plane, that is, the homography matrix between plane 1 and the original plane.

[0067] If the original plane is projected onto another plane (plane 2), it can be expressed as:

[0068]

[0069] Where (x2, y2) represents the coordinates of the corresponding point on plane 2, and H2 represents the homography matrix between plane 1 and the original plane. Due to the reversibility of the homography matrix, according to equations (6) and (7), the relationship between plane 1 and plane 2 is:

[0070]

[0071] Where H3 represents the homography matrix between plane 1 and plane 2.

[0072] The same mapping relationship also exists between the images taken before and after the camera moves, such as Figure 3 shown.

[0073] It can be expressed as:

[0074]

[0075] Among them, (u, v) and (u', v') represent the pixel coordinates of the target point before and after the camera moves; H 12 Indicates the mapping relationship between the measurement plane in the image under two viewing angles; h ij Represents the deformation characteristics between two images, i = 1, 2, 3, j = 1, 2, 3. Homography matrix H 12 There are eight unknowns in , so we need at least four pairs of corresponding point coordinates in the two images to find the homography matrix.

[0076] However, it is worth noting that the above DIC method is limited to points on the same plane. If the selected points are not on the same plane, the homography matrix solution will be wrong. Therefore, in order to accurately eliminate the error caused by the shaking of the drone, it is required to fix the calibration plate coplanar with the measurement target plane.

[0077] Specifically, the displacement measurement method based on drone images and digital images in this embodiment includes the following steps:

[0078] Step 1: Solve the coordinates of the calibration plate in the world coordinate system

[0079] 11) Fix the camera position and adjust the camera angle so that the camera can capture the target plane and calibration plate. There are no strict requirements for the camera's fixed position; it only needs to be able to capture the target plane and calibration plate. In addition, if the target plane does not have rich natural texture, spray the speckle pattern on the target plane.

[0080] 12) Temporarily fix the calibration plate on the target plane. Simple fixation is only required, such as pressing the calibration plate on the target plane with your hands. Figure 4 As shown; a fixed camera is used to capture images including the target plane and all calibration plates to obtain a first image, and then the calibration plate is removed. The calibration plate of this embodiment adopts an easily identifiable chessboard.

[0081] 13) Fix the calibration plate near the target plane to ensure that the calibration plate does not move during the measurement process, such as Figure 4 As shown; a fixed camera is used to capture an image including the target plane and all calibration plates to obtain a second image.

[0082] 14) Make X in the world coordinate system W O W Y W The plane is coplanar with the target plane, and a standard world coordinate system is obtained; the coordinates of all calibration plates in the second image in the standard world coordinate system are solved.

[0083] Specifically, change the X W O W Y W The plane is built on the measurement target plane, then formula (1) can be rewritten as:

[0084]

[0085] For the sake of convenience, the i-th column of the rotation matrix R is written as the column vector R i , then the external parameter matrix M2 can be written as:

[0086] M2=(R1 R2 R3 T) (11)

[0087] According to formula (10), the X coordinate of the imaging plane and the world coordinate system is W O W Y W The homography matrix H between planes is:

[0088] H=M1(R1 R2 T) (12)

[0089] The camera's internal parameter matrix M1 is only related to the camera itself and can be obtained after a calibration. It will not change during the measurement process, so the camera's internal parameters are known during the measurement process. Therefore, as long as the camera's external parameters are solved, the homography matrix can be calculated. Note that the obtained homography matrix is ​​the image plane and the world coordinate system X W O W Y W Therefore, in order to accurately measure the displacement of the target plane, the X coordinate system of the world coordinate system must be W O W Y W The plane is built on the measurement target plane.

[0090] From Equations (1) and (2), we can see that the extrinsic parameter matrix M2 has a total of 12 unknowns, so at least 6 pairs of points are required to solve the extrinsic parameter matrix. To this end, a prescribed calibration plate is placed near the measurement plane, and the camera extrinsic parameter matrix is ​​solved by the coordinates of the points on the calibration plate in the world coordinate system and their corresponding pixel coordinates in the image.

[0091] In actual measurement, it is often difficult to fix the calibration plate on the measurement target plane. Therefore, the coordinates of the points after the calibration plate is fixed in the world coordinate system are difficult to directly determine. The main purpose of step 1 of this embodiment is to solve the coordinates of the calibration plate in the world coordinate system.

[0092] Two calibration plates were used, such as Figure 4 As shown in the first image, the origin of the world coordinate is at the lower left corner of the left calibration plate, and X W Axis and Y W The axes are established along the two edges of the left calibration plate. At this time, the world coordinate system satisfies X W O W Y W The plane satisfies the requirement of being coplanar with the target plane. This coordinate system is called the standard world coordinate system. By identifying the points on the chessboard and the size of the chessboard, the world coordinates of the left calibration plate in the first image can be solved. Because the camera's internal parameter matrix is ​​known, only the external parameters, that is, the relationship between the camera coordinate system and the world coordinate system, are considered. Specifically, in step 14), the coordinates of the calibration plate in the second image in the standard world coordinate system are solved as follows:

[0093] When the calibration plate is temporarily fixed, the homogeneous coordinate transformation relationship between the world coordinates and the camera coordinates of the calibration plate is:

[0094]

[0095] Among them, (x w0 ,y w0 ,z w0) represents the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane; M0 represents the external parameter matrix of the camera in the standard world coordinate system; (x c0 ,y c0 ,z c0 ) represents the coordinates of the calibration plate in the camera coordinate system when it is temporarily fixed on the target plane; r ij The parameters representing the camera's posture in the standard world coordinate system, representing the degree of rotation of the camera in all directions, constitute the rotation matrix; t i Represents the position parameters of the camera in the standard world coordinate system, indicating the position of the camera in various directions, forming a position vector; i = 1, 2, 3, j = 1, 2, 3;

[0096] After the calibration plate is fixed, an auxiliary world coordinate system is created to move with the calibration plate, so that the coordinates of the calibration plate in the auxiliary world coordinate system are the same as the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane. The homogeneous coordinate transformation relationship between the coordinates of the calibration plate in the auxiliary world coordinate system and the camera coordinates is:

[0097]

[0098] Where M' represents the external parameter matrix of the camera in the auxiliary world coordinate system; (x c1 ,y c1 ,z c1 ) represents the coordinates of the calibration plate in the camera coordinate system after it is fixed; r ij The parameters representing the camera's posture in the auxiliary world coordinate system, representing the degree of rotation of the camera in each direction, constitute the rotation matrix; t i Represents the position parameters of the camera in the auxiliary world coordinate system, indicating the position of the camera in various directions, forming a position vector; i = 1, 2, 3, j = 1, 2, 3;

[0099] Since the camera is fixed during the process of capturing the first and second images, we have:

[0100]

[0101] The coordinates of the calibration plate in the standard world coordinate system after being fixed are obtained:

[0102]

[0103] Among them, (x w1 ,y w1 ,z w1 ) represents the coordinates of the calibration plate in the standard world coordinate system after it is fixed.

[0104] In this way, the same method can be used to solve the coordinates of all calibration plates in the standard coordinate system when they are temporarily fixed and after they are finally fixed.

[0105] Step 2: Measure displacement

[0106] 21) The drone carrying the camera is driven to hover at a suitable position, and the camera is used to capture the target plane and the area where all calibration plates are located to obtain a video.

[0107] 22) Convert the first image into a standard reference image, and use equations (1) and (12) to calculate the homography matrix of the standard reference image and each frame image relative to the target plane according to the coordinates and pixel coordinates of the calibration plate in the standard world coordinate system.

[0108] 23) After respectively calculating the homography matrix of the standard reference image and each frame image relative to the target plane, the homography matrix of each frame image in the video relative to the standard reference image can be calculated.

[0109] 24) Use digital image correlation method to solve the pixel displacement of the target point, and use the size factor to convert the pixel displacement into actual physical displacement.

[0110] Specifically, in this embodiment, the method of converting the first image into the standard reference image using the homography matrix is:

[0111] Let the size factor be:

[0112]

[0113] Among them, f u and f v Represent the size factors in the U direction and V direction in the pixel coordinate system respectively; l1 represents the distance between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; l2 represents the distance between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed; (Δu1, Δv1) represents the pixel coordinate difference between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; (Δu2, Δv2) represents the pixel coordinate difference between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed;

[0114] Define a reference point on the first image so that the pixel coordinates (u0, v0) of the reference point in the standard reference image are the same as those in the first image. Calculate the pixel coordinates of any point in the standard reference image as follows:

[0115]

[0116] Among them, (u i ,v i ) represents the pixel coordinates of any point in the standard reference image, (Δxi ,Δy i ) represents any point (u i ,v i ) and the coordinate difference of the reference point (u0, v0) in the standard world coordinate system;

[0117] Then the transformation relationship between the first image and the standard reference image is:

[0118]

[0119] Where (u, v) represents the pixel coordinates of a point in the standard reference image; (u1, v1) represents the pixel coordinates of the point in the first image corresponding to the point (u, v) in the standard reference image; H′ represents the homography matrix between the first image and the standard reference image; h ij Parameters representing image deformation characteristics, i = 1, 2, 3, j = 1, 2, 3;

[0120] Since the homography matrix H′ has 8 unknowns, the pixel coordinates of the corresponding 4 pairs of points in the first image and the standard reference image are used to solve the unknowns in the homography matrix H′, and the homography matrix H′ between the first image and the standard reference image is obtained. The homography matrix H′ is used to convert the first image into the standard reference image.

[0121] At this point, all images meet the DIC requirement of "the camera is fixed and the principal optical axis is perpendicular to the target plane". On this basis, the DIC algorithm is used to solve the target point displacement.

[0122] Specifically, such as Figure 4 As shown, only two calibration plates are used in this embodiment. In some other embodiments, the number of calibration plates may be 1, 3, or more than 3. The more calibration plates there are and the more evenly they are distributed, the higher the calculation result will be.

[0123] The above-described embodiments are merely preferred embodiments for the purpose of fully illustrating the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are within the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

Claims

1. A displacement measurement method based on drone images and digital images, characterized by: The steps include: Step 1: Solve the coordinates of the calibration plate in the world coordinate system 11) Fix the camera position and adjust the camera angle so that the camera can capture the target plane and calibration plate; 12) temporarily fixing the calibration plate on the target plane, and using a fixed camera to capture an image including the target plane and all calibration plates to obtain a first image; 13) Fixing the calibration plate near the target plane, and using a fixed camera to capture an image including the target plane and all calibration plates to obtain a second image; 14) Make X in the world coordinate system W O W Y W The plane is coplanar with the target plane, and a standard world coordinate system is obtained; the coordinates of all calibration plates in the second image in the standard world coordinate system are solved; Step 2: Measure displacement 21) Drive the drone carrying the camera to hover at a suitable position, and use the camera to shoot the area including the target plane and all calibration plates to obtain a video; 22) converting the first image into a standard reference image, and calculating the homography matrix of the standard reference image and each frame image relative to the target plane according to the coordinates and pixel coordinates of the calibration plate in the standard world coordinate system; 23) Calculate the homography matrix of each frame image in the video relative to the standard reference image; 24) Use digital image correlation method to solve the pixel displacement of the target point, and use the size factor to convert the pixel displacement into actual physical displacement.

2. The displacement measurement method based on drone images and digital images according to claim 1, characterized in that: In step 14), the coordinates of the calibration plate in the second image in the standard world coordinate system are solved as follows: When the calibration plate is temporarily fixed, the homogeneous coordinate transformation relationship between the world coordinates and the camera coordinates of the calibration plate is: Among them, (x w0 ,y w0 ,z w0 ) represents the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane; M0 represents the external parameter matrix of the camera in the standard world coordinate system; (x c0 ,y c0 ,z c0 ) represents the coordinates of the calibration plate in the camera coordinate system when it is temporarily fixed on the target plane; r ij Parameters representing the camera's posture in the standard world coordinate system; t i Represents the position parameters of the camera in the standard world coordinate system; i = 1, 2, 3, j = 1, 2, 3; After the calibration plate is fixed, an auxiliary world coordinate system is created to move with the calibration plate, so that the coordinates of the calibration plate in the auxiliary world coordinate system are the same as the coordinates of the calibration plate in the standard world coordinate system when it is temporarily fixed on the target plane. The homogeneous coordinate transformation relationship between the coordinates of the calibration plate in the auxiliary world coordinate system and the camera coordinates is: Where M' represents the external parameter matrix of the camera in the auxiliary world coordinate system; (x c1 ,y c1 ,z c1 ) represents the coordinates of the calibration plate in the camera coordinate system after it is fixed; r′ ij Parameters representing the camera's posture in the auxiliary world coordinate system; t' i Represents the position parameters of the camera in the auxiliary world coordinate system; i = 1, 2, 3, j = 1, 2, 3; Since the camera is fixed during the process of capturing the first and second images, we have: The coordinates of the calibration plate in the standard world coordinate system after being fixed are obtained: Among them, (x w1 ,y w1 ,z w1 ) represents the coordinates of the calibration plate in the standard world coordinate system after it is fixed.

3. The displacement measurement method based on drone images and digital images according to claim 1 is characterized in that: In step 22), the method of converting the first image into a standard reference image using the homography matrix is: Let the size factor be: Among them, f u and f v Represent the size factors in the U direction and V direction in the pixel coordinate system respectively; l1 represents the distance between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; l2 represents the distance between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed; (Δu1, Δv1) represents the pixel coordinate difference between the upper left corner and the lower right corner of the calibration plate when it is temporarily fixed; (Δu2, Δv2) represents the pixel coordinate difference between the lower left corner and the upper right corner of the calibration plate when it is temporarily fixed; Define a reference point on the first image so that the pixel coordinates (u0, v0) of the reference point in the standard reference image are the same as those in the first image. Calculate the pixel coordinates of any point in the standard reference image as follows: Among them, (u i ,v i ) represents the pixel coordinates of any point in the standard reference image, (Δx i ,Δy i ) represents any point (u i ,v i ) and the reference point (u0, v0) in the standard world coordinate system; and: Where (u, v) represents the pixel coordinates of a point in the standard reference image; (u1, v1) represents the pixel coordinates of the point in the first image corresponding to the point (u, v) in the standard reference image; H′ represents the homography matrix between the first image and the standard reference image; h ij Represents the deformation characteristics of the image, i = 1, 2, 3, j = 1, 2, 3; Since the homography matrix H′ has 8 unknowns, the pixel coordinates of the corresponding 4 pairs of points in the first image and the standard reference image are used to solve the unknowns in the homography matrix H′, and the homography matrix H′ between the first image and the standard reference image is obtained. The homography matrix H′ is used to convert the first image into the standard reference image.

Images