A method and device for monitoring slope displacement based on binocular vision of a drone

Abstract

The present application relates to the technical field of slope displacement monitoring, in particular to a binocular vision slope displacement monitoring method and device based on a UAV. After the UAV is accurately positioned by monitoring the auxiliary positioning mark target and the UAV positioning mark target, the three-dimensional space coordinate information of the monitoring mark target is obtained by using the original image shot by the UAV, and then the slope displacement is obtained according to the three-dimensional space coordinate information of the monitoring mark target, so as to realize the monitoring of the slope displacement. In this way, the UAV positioning and shooting need only be remotely controlled, and the field work intensity is low. At the same time, the device is simple, the purchase cost and the equipment maintenance cost are low, and the degree of automation is high. Furthermore, the three-dimensional space coordinate information of the monitoring mark target in the image is obtained by using the binocular positioning method, and then the slope displacement is obtained, so the accuracy is high and meets the requirements of engineering practice. In this way, the requirements of low equipment maintenance cost, slope measurement accuracy meeting the requirements of engineering practice and low field work intensity are met at the same time, and the benefit is higher.

Description

Technical Field

[0001] This invention relates to the field of slope displacement monitoring technology, specifically to a binocular vision slope displacement monitoring method and device based on unmanned aerial vehicles (UAVs). Background Technology

[0002] During the daily use of highways, the stability of slopes directly affects their safety, and consequently, the safe use of the highway. Slope stability is directly reflected in displacement and deformation; therefore, accurately monitoring slope displacement and deformation information is crucial for assessing slope safety. Common methods for slope displacement monitoring include instrument monitoring, GPS satellite measurement, lidar scanning, and video image monitoring.

[0003] However, in existing technologies, instrument monitoring methods can only monitor displacement at fixed locations. If the monitoring location is changed, the high slope displacement monitoring device needs to be re-fixed, resulting in significant fieldwork and inconvenience for users. GPS satellite measurement methods have large errors, with the maximum error reaching the meter level, and the accuracy cannot meet the needs of engineering applications. LiDAR scanning methods are not only expensive, but also require scanning each target with a laser, which is time-consuming. Video image monitoring methods require several fixed cameras for image acquisition, resulting in high deployment and maintenance costs. Therefore, to simultaneously meet the requirements of low equipment maintenance costs, slope measurement accuracy that meets engineering requirements, and low fieldwork intensity, a more efficient slope displacement monitoring method is needed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a binocular vision-based slope displacement monitoring method based on unmanned aerial vehicles (UAVs), which can simultaneously meet the requirements of monitoring accuracy, low cost, and low fieldwork intensity, thereby improving efficiency.

[0005] In a first aspect, the present invention provides a binocular vision method for monitoring slope displacement based on unmanned aerial vehicles (UAVs).

[0006] In the first feasible approach, a UAV-based binocular vision method for slope displacement monitoring includes:

[0007] In the environment of the slope to be measured, monitoring target points, monitoring auxiliary positioning target points, and UAV positioning target points are set up.

[0008] The drone is located based on the monitoring-assisted positioning target points and the drone positioning target points;

[0009] The system acquires the original images captured after the drone is located, and then enhances these images to obtain the monitoring images.

[0010] The binocular positioning method was used to obtain the three-dimensional spatial coordinate information of the monitoring target points in the monitoring images;

[0011] The slope displacement is obtained based on the three-dimensional spatial coordinate information of the monitoring target points and the initial coordinate information of the monitoring target points.

[0012] In conjunction with the first feasible method, the second feasible method involves deploying monitoring marker targets, monitoring auxiliary positioning marker targets, and UAV positioning marker targets in the environment of the slope to be measured, including:

[0013] Determine the environmental characteristics of the slope to be measured, including the shape and location of the slope;

[0014] Select the location of the UAV positioning marker target point and the location of multiple auxiliary positioning marker target points based on environmental characteristics;

[0015] The location of the monitoring target points is determined based on each auxiliary positioning target point, and each target point is deployed by spraying paint.

[0016] In the third feasible method, in conjunction with the first feasible method, the UAV is located based on the monitoring-assisted positioning target point and the UAV positioning target point, including:

[0017] Control the drone to fly vertically upwards from the drone positioning marker target point;

[0018] When the drone's two cameras cover all the monitoring-aided positioning target points, it is determined that the drone has completed the initial positioning.

[0019] Acquire positioning images taken by the drone after initial positioning;

[0020] The drone is located based on the positioning image.

[0021] In conjunction with the third feasible method, the fourth feasible method involves image localization of the UAV based on the positioning image, including:

[0022] Obtain the current position of the target point for monitoring auxiliary positioning in the positioning image;

[0023] Determine whether the current position of the monitoring auxiliary positioning marker target point is consistent with the initial position of the monitoring auxiliary positioning marker target point;

[0024] If the current position is inconsistent with the initial position, the drone is adjusted until the current position of the monitoring auxiliary positioning target point is consistent with the initial position.

[0025] In conjunction with the first feasible method, the fifth feasible method involves enhancing the original image to obtain a monitoring image, including:

[0026] The image enhancement method using image histogram specification mapping is employed to enhance the original image, resulting in an enhanced image.

[0027] Obtain the transformation matrices corresponding to the two cameras of the drone, and use the transformation matrices as correction parameters;

[0028] The enhanced image is subjected to distortion correction processing based on the correction parameters to obtain the monitoring image.

[0029] In conjunction with the fifth feasible method, the sixth feasible method employs an image histogram specification mapping enhancement method to enhance the original image, obtaining an enhanced image, including:

[0030] The original image is processed to grayscale to obtain a grayscale statistical histogram;

[0031] Obtain the gray levels of the gray-level statistical histogram and normalize each gray level of the gray-level statistical histogram.

[0032] The original image is processed to obtain a standardized image;

[0033] Obtain the gray levels of the defined image and normalize each gray level of the defined image;

[0034] Obtain the mapping relationship between the gray levels of the original image and the gray levels of the defined image;

[0035] Based on the mapping relationship, obtain the standardized histogram corresponding to the gray-level statistical histogram, and determine the standardized histogram as the enhanced image.

[0036] In the seventh feasible method, in conjunction with the first feasible method, a binocular positioning method is used to obtain the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image, including:

[0037] The pixel coordinates of each monitoring target point in the monitoring image are obtained using a lightweight YOLOv6 model.

[0038] The pixel coordinates of each monitoring target point are converted into three-dimensional spatial coordinate information according to the spatial coordinate transformation formula.

[0039] Combining the seventh feasible method, the spatial coordinate transformation formula in the eighth feasible method is:

[0040]

[0041] In the above formula, X, Y, and Z are the three-dimensional spatial coordinates of the monitored target point, D is the distance between the focal points of the two cameras on the UAV, f is the focal length of the camera, the coordinates of the same feature point of the spatial object viewed by the two cameras at the same time are P(x, y, z), and the projected coordinates of point P on the left camera are P0.left (x l ,y l The projected coordinates of point P on the right-hand camera are P_0. right (x r ,y r ).

[0042] In the ninth feasible method, in conjunction with the first feasible method, the slope displacement is obtained based on the three-dimensional spatial coordinate information and the initial coordinate information of the monitoring target point, including:

[0043] The displacement changes of each monitoring target point are obtained based on the three-dimensional spatial coordinate information and the initial coordinate information of each monitoring target point.

[0044] The slope displacement is obtained based on the displacement changes of each monitoring target point.

[0045] Secondly, the present invention provides a binocular vision slope displacement monitoring device based on unmanned aerial vehicles (UAVs).

[0046] In the tenth feasible method, a binocular vision slope displacement monitoring device based on an unmanned aerial vehicle (UAV) includes:

[0047] Each marker target deployment module is used to deploy monitoring marker targets, monitoring auxiliary positioning marker targets, and UAV positioning marker targets in the environment of the slope to be measured;

[0048] The UAV positioning module is used to locate the UAV based on the monitoring auxiliary positioning target point and the UAV positioning target point;

[0049] The monitoring image acquisition module is used to acquire the original images taken by the UAV after positioning, and to enhance the original images to obtain the monitoring images;

[0050] The three-dimensional spatial coordinate information acquisition module is used to acquire the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image using the binocular positioning method.

[0051] The slope displacement acquisition module is used to acquire slope displacement based on the three-dimensional spatial coordinate information and the initial coordinate information of the monitoring target points.

[0052] As can be seen from the above technical solution, the beneficial technical effects of the present invention are as follows:

[0053] After accurately locating the drone using monitoring-assisted positioning markers and drone positioning markers, the three-dimensional spatial coordinates of the monitoring markers are obtained from the raw images captured by the drone. This information is then used to determine the slope displacement, thus enabling slope displacement monitoring. This method requires only remote control of the drone for positioning and image capture, reducing fieldwork intensity. Furthermore, the equipment is simple, with low purchase and maintenance costs and a high degree of automation. The use of binocular positioning to obtain the three-dimensional spatial coordinates of the monitoring markers in the images, and subsequently the slope displacement measurement, provides high accuracy, meeting engineering practice requirements. This approach simultaneously satisfies the requirements of low equipment maintenance costs, slope measurement accuracy meeting engineering practice requirements, and low fieldwork intensity, resulting in higher efficiency and greater applicability. Attached Figure Description

[0054] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0055] Figure 1 A schematic diagram of a binocular vision slope displacement monitoring method based on unmanned aerial vehicles (UAVs) provided in this embodiment;

[0056] Figure 2 This is a schematic diagram of a monitoring marker target pattern provided in this embodiment;

[0057] Figure 3 This is a schematic diagram of a drone positioning target pattern provided in this embodiment;

[0058] Figure 4 This is a schematic diagram of a monitoring-aided positioning marker target pattern provided in this embodiment;

[0059] Figure 5 This is a plan view of the deployment of monitoring target points and auxiliary monitoring target points on the slope surface provided in this embodiment;

[0060] Figure 6 This is a schematic diagram showing the spacing of the monitoring marker target points provided in this embodiment;

[0061] Figure 7 This is a schematic diagram illustrating the initial positioning of a drone in this embodiment;

[0062] Figure 8 This embodiment provides a schematic diagram of binocular vision three-dimensional spatial positioning.

[0063] Figure 9This embodiment provides a schematic diagram of the structure of a binocular vision slope displacement monitoring device based on an unmanned aerial vehicle (UAV).

[0064] Figure label:

[0065] 1-UAV, 2-Monitoring auxiliary positioning target point, 3-Monitoring target point, 4-UAV positioning target point, 5-Slope to be measured, 6-Roadway. Detailed Implementation

[0066] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0067] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art to which this invention pertains. The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data used can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. Unless otherwise stated, the term "a plurality of" means two or more. In this disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B means: A or B. The term "and / or" describes an association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B. The term "corresponding" can refer to an association or binding relationship; A corresponding to B means that there is an association or binding relationship between A and B.

[0068] Combination Figure 1 As shown, this embodiment provides a binocular vision-based slope displacement monitoring method based on unmanned aerial vehicles (UAVs), including:

[0069] Step S01: Deploy monitoring target points, monitoring auxiliary positioning target points, and UAV positioning target points in the environment of the slope to be measured;

[0070] Step S02: Locate the UAV based on the monitoring-assisted positioning target point and the UAV positioning target point;

[0071] Step S03: Obtain the original image captured by the UAV after positioning, and perform enhancement processing on the original image to obtain the monitoring image;

[0072] Step S04: Use binocular positioning to obtain the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image;

[0073] Step S05: Obtain the slope displacement based on the three-dimensional spatial coordinate information of the monitoring target point and the initial coordinate information of the monitoring target point.

[0074] After accurately locating the drone using monitoring-assisted positioning markers and drone positioning markers, the three-dimensional spatial coordinates of the monitoring markers are obtained from the raw images captured by the drone. This information is then used to determine the slope displacement, thus enabling slope displacement monitoring. This method requires only remote control of the drone for positioning and image capture, reducing fieldwork intensity. Furthermore, the equipment is simple, with low purchase and maintenance costs and a high degree of automation. The use of binocular positioning to obtain the three-dimensional spatial coordinates of the monitoring markers in the images, and subsequently the slope displacement measurement, provides high accuracy, meeting engineering practice requirements. This approach simultaneously satisfies the requirements of low equipment maintenance costs, slope measurement accuracy meeting engineering practice requirements, and low fieldwork intensity, making it more suitable for various applications.

[0075] Optionally, monitoring target points, auxiliary positioning target points, and UAV positioning target points are set up in the environment of the slope to be measured, including: determining the environmental characteristics of the slope to be measured, including the shape and location of the slope; selecting the location of UAV positioning target points and multiple auxiliary positioning target points based on the environmental characteristics; determining the location of monitoring target points based on each auxiliary positioning target point; and setting up each target point by spraying paint.

[0076] In some embodiments, a first preset pattern is sprayed at a predetermined distance from the slope to be measured, or on the other side of the roadway corresponding to the slope to be measured, at a location with solid geology, low slippage, unobstructed ground, and open view. This sprayed location is then used as a target point for UAV positioning. The shape of the first preset pattern is as follows: Figure 2 As shown.

[0077] In some embodiments, multiple relatively stable areas are selected around the slope to be measured. These areas, when connected, completely enclose the slope. A second preset pattern is sprayed onto these areas as monitoring-aided positioning markers. If there are no relatively stable areas or insufficient areas around the slope, structurally stable marker posts are poured, and the second preset pattern is sprayed onto the marker posts. The shape of the second preset pattern is as follows: Figure 3 As shown.

[0078] In some embodiments, monitoring auxiliary positioning target points are connected to form a square area, and a third preset pattern is uniformly sprayed within the square area at preset intervals as monitoring target points. The shape of the third preset pattern is as follows: Figure 4 As shown. The relative positions of the monitoring target points and the monitoring auxiliary positioning target points on the plane of the slope to be measured are as follows. Figure 5As shown, the monitoring target points are black and red alternating cross markers arranged in an n×n square matrix; the auxiliary monitoring target points are black and white alternating diagonal cross markers, placed in the edge area of ​​the slope to be measured. The spacing between the monitoring target points is as follows. Figure 6 As shown, Δx is the horizontal spacing between each monitoring target point, and Δy is the vertical spacing.

[0079] In some embodiments, slope control surveying should select high-precision, low-error, and easy-to-implement technical methods that match UAV monitoring methods. Total stations can be used to determine monitoring and identification target points. For some steep slopes, a combination of multiple technologies can be used.

[0080] In some embodiments, each target point has the characteristics of relatively regular patterns, high recognizability, strong color penetration, good anti-interference, and good stability. Each preset pattern is selected from those that form a clear contrast with the color of the slope surface to be measured and are conducive to the extraction of the target points. For example... Figure 3 , Figure 4 and Figure 5 As shown, alternating use of two high-contrast colors ensures strong differentiation and high visibility of the identified target points. The regular square pattern helps improve the detection accuracy during target recognition. In some embodiments, the first preset pattern, the second preset pattern, and the third preset pattern are respectively... Figure 3 , Figure 4 and Figure 5 The first, second, and third preset patterns are all different in shape. Figure 3 and Figure 5 The pattern in the picture is red and black.

[0081] Optionally, the UAV is positioned based on the monitoring-aided positioning target point and the UAV positioning target point, including: controlling the UAV to fly vertically upward from the UAV positioning target point; determining that the UAV has completed the initial positioning when the field of view of the two cameras of the UAV covers all the monitoring-aided positioning target points; acquiring the positioning image taken by the UAV after the initial positioning; and performing image positioning of the UAV based on the positioning image.

[0082] Combination Figure 7As shown, in some embodiments, four monitoring auxiliary positioning target points 2 are arranged on the slope 5 to be measured. Multiple monitoring target points 3 are evenly distributed within the area connected to the four monitoring auxiliary positioning target points 2. On the other side of the roadway 6 corresponding to the slope to be measured, a drone positioning target point 4 is arranged. The drone 1 flies vertically upward from the drone positioning target point 4 until the field of view of the drone's two cameras completely covers the monitoring auxiliary positioning target point, achieving preliminary three-dimensional spatial positioning of the drone. During the monitoring process, it is ensured that the drone's status is stable and all parameters are normal, enabling real-time monitoring of the slope condition. During the shooting process, it is ensured that the photos taken by the drone are clear and that the camera's field of view covers the entire area.

[0083] Optionally, the UAV is image-localized based on the positioning image, including: obtaining the current position of the monitoring auxiliary positioning target point in the positioning image; determining whether the current position of the monitoring auxiliary positioning target point is consistent with the initial position of the monitoring auxiliary positioning target point; and adjusting the UAV if the current position is inconsistent with the initial position until the current position of the monitoring auxiliary positioning target point is consistent with the initial position.

[0084] Optionally, the initial position is the location of the target point in the original image when the UAV last captured the original image, as determined by the monitoring-assisted positioning marker.

[0085] In some embodiments, after the UAV completes the initial positioning, the UAV takes a picture of the current original image and identifies the current position of the monitoring auxiliary positioning target point in the current original image. The current position is compared with the initial position when the UAV last took the original image. If they are inconsistent, they are finely adjusted according to the initial position so that the current position of the monitoring auxiliary positioning target point is completely consistent with the initial position. This ensures that the UAV's position in three-dimensional space does not shift between the two monitoring sessions, reduces the impact of changes in the UAV's three-dimensional spatial position on the slope monitoring displacement results, and improves the accuracy of slope displacement monitoring.

[0086] Optionally, the original image is enhanced to obtain a monitoring image, including: performing image enhancement processing on the original image using an image histogram-defined mapping enhancement method to obtain an enhanced image; obtaining the transformation matrix corresponding to the two cameras of the UAV and using the transformation matrix as a correction parameter; and performing distortion correction processing on the enhanced image according to the correction parameter to obtain a monitoring image.

[0087] Optionally, an image enhancement process is performed on the original image using an image histogram specification mapping enhancement method to obtain an enhanced image. This includes: processing the original image in grayscale to obtain a grayscale statistical histogram; obtaining the grayscale levels of the grayscale statistical histogram and normalizing each grayscale level; specification processing the original image to obtain a specification image; obtaining the grayscale levels of the specification image and normalizing each grayscale level; obtaining the mapping relationship between the grayscale levels of the original image and the grayscale levels of the specification image; obtaining the specification histogram corresponding to the grayscale statistical histogram based on the mapping relationship, and determining the specification histogram as the enhanced image.

[0088] Optionally, the expression for the normalized probability of the gray-scale statistical histogram is:

[0089] p s (s i ) = n i / N

[0090] Among them, s i Let s be the normalized gray value of the i-th gray level. i =i / 255, n i The total number of pixels at each gray level in the grayscale histogram is N, where N is the total number of pixels in the image.

[0091] Optionally, the expression for the normalized probability of each gray level of the image is defined as follows:

[0092] p t (t j ) = n j / N

[0093] Among them, t j Let t be the normalized gray value of the j-th gray level. j =j / 255,n j N is the total number of pixels used to define the gray levels of an image, where N is the total number of pixels in the image.

[0094] Optionally, the relationship between the original histogram and the target histogram is represented by an integer function, that is, mapping from the target cumulative histogram to the original cumulative histogram, finding a set of mappings for each target cumulative histogram. The integer function H(l) has elements satisfying the relationship: 0 ≤ H(0) ≤ H(1) ≤ … ≤ H(J-1) ≤ I-1, where J is the number of gray levels in the standardized image, I is the number of gray levels in the original image, and I ≥ J. 0 represents the minimum value for each gray level mapping.

[0095] Optionally, the mapping relationship between the gray levels of the original image and the gray levels of the defined image is expressed by the following formula:

[0096]

[0097] In some embodiments, according to the group mapping rule, when l=0, the following will be implemented: p s (s i Mapping to p t (t0); when l≥1, p s (s i Mapping to p t (t l The standardized histogram obtained through the mapping relationship is used to enhance the pixels at the corresponding levels of the original image, thereby amplifying the differences between the features of different objects in the image and highlighting the features of each target point.

[0098] In some embodiments, the original cumulative histogram probabilities are: 0.1, 0.15, 0.3, 0.5, 0.7, 0.85, 0.9, 1. The target cumulative histogram probabilities are: 0.3, 0.75, 1. The mapping relationship is obtained according to the formula expressing the mapping relationship between the gray levels of the original image and the gray levels of the specified image. The corresponding mapping relationships are: 0.1, 0.15, 0.3, 0.5 → 0.3; 0.7, 0.85 → 0.75; 0.9, 1 → 1. Finally, the specified histogram, i.e., the target histogram, is obtained by summing up the mapping quantities. In this way, the specified histogram statistically and unbiasedly maps each gray level of the original image to the specified gray levels, thereby achieving image enhancement and improving the accuracy of slope displacement monitoring.

[0099] Optionally, the transformation matrices corresponding to the two cameras of the drone include intrinsic parameter matrices and extrinsic parameter matrices.

[0100] Optionally, the intrinsic parameter matrix can be obtained from the affine transformation matrix and the perspective projection matrix, calculated as follows:

[0101]

[0102] Where f is the camera focal length, dX represents the physical length of a pixel in the X-direction image captured by the camera, and dY represents the physical length of a pixel in the Y-direction image captured by the camera. u0 and v0 represent the camera coordinates in the pixel coordinate system, and θ represents the angle between the horizontal and vertical edges of the camera's image sensor.

[0103] Optionally, the extrinsic parameter matrix determines the relative positional relationship between the camera image pixel coordinates and the three-dimensional spatial coordinates. The expression for the extrinsic parameter matrix is ​​as follows:

[0104]

[0105] Where R represents the rotation matrix and T represents the translation vector.

[0106] Optionally, the intrinsic and extrinsic parameter matrices in the transformation matrix can be used as correction parameters. The relationship formula of the camera image distortion correction model is obtained using the correction parameters as shown below:

[0107]

[0108] Where X, Y, and Z are the three-dimensional spatial coordinates of the target point, and u and v are the pixel coordinates of the target point on the image plane.

[0109] In some embodiments, through image parameter calibration experiments, the corresponding values ​​of the intrinsic and extrinsic parameter matrices of each camera are obtained according to the camera image distortion correction model relationship formula, i.e., R, T, dX, and dY are obtained. Then, each pixel in the slope field image is substituted into the correction model relationship formula to obtain the corrected target image.

[0110] In some embodiments, the parameter values ​​of the intrinsic and extrinsic parameter matrices of each camera are solved through image parameter calibration experiments. Then, the intrinsic and extrinsic parameter matrices are substituted into the distortion correction model relationship formula to achieve distortion correction processing of the enhanced image.

[0111] Optionally, a binocular positioning method is used to obtain the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image, including: using a lightweight YOLOv6 model to obtain the pixel coordinates of each monitoring target point in the monitoring image; and converting the pixel coordinates of each monitoring target point into three-dimensional spatial coordinate information according to the spatial coordinate transformation formula.

[0112] In some embodiments, the lightweight YOLOv6 model's network architecture involves replacing its backbone network with Mobilenetv3, which is more suitable for drones, and fusing it with PANet. Image data is fed into Mobilenetv3, and the output of Mobilenetv3 is then fed into the PANet network for hierarchical convolution to extract feature maps at different depths. Simultaneously, the hyperparameters of the PANet network are adjusted to further reduce the model size, enabling the drone to better extract relevant feature information.

[0113] In some embodiments, the monitoring image is input into a lightweight YOLOv6 model to segment and extract the pixel regions of each monitoring target point, thereby obtaining the pixel coordinates of the monitoring target points.

[0114] Optionally, the lightweight YOLOv6 model can be deployed on a drone, which enables rapid location of monitoring target points on the slope and real-time and efficient identification of the displacement of the slope under test. This facilitates timely warnings when the slope displacement exceeds the warning threshold, thereby improving roadway safety.

[0115] Optionally, the drone is equipped with two cameras, an image processing device, a processor, and an information transmission device. The image processing device acquires monitoring images based on the raw images. The processor uses a lightweight YOLOv6 model to identify the coordinates of monitoring markers and calculates slope displacement. The information transmission device transmits the data to a cloud server. This allows for direct identification of slope displacement by the drone when it is significant, facilitating timely and efficient early warning. For smaller slope displacements, the server is used for identification.

[0116] Alternatively, the spatial coordinate transformation formula is:

[0117]

[0118] In the above formula, X, Y, and Z are the three-dimensional spatial coordinates of the monitored target point, D is the distance between the focal points of the two cameras on the UAV, f is the focal length of the camera, and the coordinates of the same feature point P of the spatial object viewed by the two cameras at the same time are (x, y, z). The projected coordinates of point P on the left camera are P0. left (x l ,y l The projected coordinates of point P on the right-hand camera are P_0. right (x r ,y r ).

[0119] Combination Figure 8 As shown, in some embodiments, the left and right cameras on the drone capture images on the left and right sides, respectively. The distance between the line connecting the focal point of the left camera in the left image and the focal point of the right camera in the right image is D. The projection of feature point P(x, y, z) in the left image is P. left (x l ,y l The projection of feature point P(x, y, z) onto the left image is P. right (x) r ,y r ).

[0120] Optionally, the slope displacement is obtained based on the three-dimensional spatial coordinate information and the initial coordinate information of the monitoring target points, including: obtaining the displacement change of each monitoring target point based on the three-dimensional spatial coordinate information and the initial coordinate information of each monitoring target point; and obtaining the slope displacement based on the displacement change of each monitoring target point.

[0121] Alternatively, the slope displacement can be obtained using the following formula:

[0122]

[0123] Among them, the three-dimensional spatial coordinates of each monitoring target point are (xi ′,y i ′,z i The initial coordinates of each monitoring target point are (x'), and (x''). i ,y i ,z i ), where i represents the i-th monitoring and identification target point, and i and n are both positive integers.

[0124] Combination Figure 9 As shown, a binocular vision slope displacement monitoring device based on unmanned aerial vehicles (UAVs) includes:

[0125] Each marker target deployment module 101 is used to deploy monitoring marker targets, monitoring auxiliary positioning marker targets, and UAV positioning marker targets in the environment of the slope to be measured;

[0126] The UAV positioning module 102 is used to locate the UAV based on the monitoring auxiliary positioning target point and the UAV positioning target point;

[0127] The monitoring image acquisition module 103 is used to acquire the original image taken by the UAV after positioning, and to enhance the original image to obtain the monitoring image;

[0128] The three-dimensional spatial coordinate information acquisition module 104 is used to acquire the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image using the binocular positioning method;

[0129] The slope displacement acquisition module 105 is used to acquire slope displacement based on the three-dimensional spatial coordinate information of the monitoring target point and the initial coordinate information of the monitoring target point.

[0130] In some embodiments, small unmanned aerial vehicles (UAVs) are increasingly being used for surface observation, and computational vision technology has also developed rapidly. Compared with traditional methods, the UAV-based binocular vision slope displacement monitoring method provided in this solution has many advantages: 1. More flexible operation mode and less fieldwork intensity; 2. Not subject to range control, non-contact, and able to dynamically reflect the characteristics of slope deformation; 3. Less susceptible to interference from ambient light, with high applicability to slope displacement monitoring; 4. Able to calculate the displacement of slope deformation more accurately with smaller errors.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for monitoring slope displacement based on binocular vision using unmanned aerial vehicles (UAVs), characterized in that, include: In the environment of the slope to be measured, monitoring target points, monitoring auxiliary positioning target points, and UAV positioning target points are set up. Control the drone to fly vertically upwards from the drone positioning target point; When the drone's two cameras cover all the monitoring-aided positioning target points, it is determined that the drone has completed the initial positioning. Acquire positioning images taken by the drone after initial positioning; Obtain the current position of the monitoring auxiliary positioning marker target point in the positioning image; Determine whether the current position of the monitoring auxiliary positioning target point is consistent with the initial position of the monitoring auxiliary positioning target point; If the current position is inconsistent with the initial position, the UAV is adjusted until the current position of the monitoring auxiliary positioning marker target point is consistent with the initial position; The initial position is the location of the target point in the original image when the drone last captured the original image; The original image captured by the UAV after positioning is obtained, and the image enhancement process is performed on the original image using the image histogram specification mapping enhancement method to obtain the enhanced image; Obtain the transformation matrices corresponding to the two cameras of the drone, and use the transformation matrices as correction parameters; The enhanced image is subjected to distortion correction processing based on the correction parameters to obtain the monitoring image; The step of performing distortion correction processing on the enhanced image according to the correction parameters to obtain the monitoring image includes: in, , , The three-dimensional spatial coordinates of the target point, , The pixel coordinates of the target point on the image plane; The binocular positioning method is used to obtain the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image; The slope displacement is obtained based on the three-dimensional spatial coordinate information of the monitoring target point and the initial coordinate information of the monitoring target point.

2. The method according to claim 1, characterized in that, The monitoring target points, monitoring auxiliary positioning target points, and UAV positioning target points are deployed in the environment of the slope to be measured, including: Determine the environmental characteristics of the slope to be measured, including the shape and location of the slope; Based on the environmental characteristics, select the location of the UAV positioning marker target point and the locations of multiple auxiliary positioning marker target points; The location of the monitoring target point is determined based on each auxiliary positioning target point, and each target point is deployed by spraying paint.

3. The method according to claim 1, characterized in that, The original image is enhanced using an image histogram-defined mapping enhancement method to obtain an enhanced image, including: The original image is processed to obtain a grayscale statistical histogram. Obtain the gray levels of the gray-level statistical histogram and normalize each gray level of the gray-level statistical histogram. The original image is then processed to obtain a standardized image; Obtain the gray levels of the defined image and normalize each gray level of the defined image; Obtain the mapping relationship between the gray levels of the original image and the gray levels of the defined image; The specified histogram corresponding to the gray-level statistical histogram is obtained according to the mapping relationship, and the specified histogram is determined as the enhanced image.

4. The method according to claim 1, characterized in that, The three-dimensional spatial coordinate information of the monitoring target points in the monitoring image is obtained using a binocular positioning method, including: The pixel coordinates of each monitoring target point in the monitoring image are obtained using a lightweight YOLOv6 model. The pixel coordinates of each monitoring target point are converted into three-dimensional spatial coordinate information according to the spatial coordinate transformation formula.

5. The method according to claim 4, characterized in that, The spatial coordinate transformation formula is as follows: In the above formula, , , To monitor the three-dimensional spatial coordinates of the identified target points, The distance between the lines connecting the focal points of the two cameras on the drone. Let be the focal length of the camera. The coordinates of the same feature point of a spatial object viewed by both cameras at the same time are . On the left side of the camera Projected coordinates of a point On the right-hand camera Projected coordinates of a point .

6. The method according to claim 1, characterized in that, The slope displacement is obtained based on the three-dimensional spatial coordinate information and the initial coordinate information of the monitoring target point, including: The displacement changes of each monitoring target point are obtained based on the three-dimensional spatial coordinate information and the initial coordinate information of each monitoring target point. The slope displacement is obtained based on the displacement changes of each monitoring target point.

7. A binocular vision slope displacement monitoring device based on unmanned aerial vehicles (UAVs), characterized in that, include: Each marker target deployment module is used to deploy monitoring marker targets, monitoring auxiliary positioning marker targets, and UAV positioning marker targets in the environment of the slope to be measured; The drone positioning module is used to control the drone to fly vertically upwards from the drone positioning target point; determine that the drone has completed preliminary positioning when the field of view of the drone's two cameras covers all the monitoring auxiliary positioning target points; acquire a positioning image taken by the drone after preliminary positioning; acquire the current position of the monitoring auxiliary positioning target points in the positioning image; determine whether the current position of the monitoring auxiliary positioning target points is consistent with the initial position of the monitoring auxiliary positioning target points; if the current position is inconsistent with the initial position, adjust the drone until the current position of the monitoring auxiliary positioning target points is consistent with the initial position; The initial position is the location of the target point in the original image when the drone last captured the original image; The monitoring image acquisition module is used to acquire the original image captured by the UAV after positioning, and to perform image enhancement processing on the original image using the image histogram specification mapping enhancement method to obtain an enhanced image; and to acquire the transformation matrix corresponding to the two cameras of the UAV, and to use the transformation matrix as a correction parameter; The enhanced image is subjected to distortion correction processing based on the correction parameters to obtain the monitoring image; The three-dimensional spatial coordinate information acquisition module is used to acquire the three-dimensional spatial coordinate information of the monitoring target points in the monitoring image using a binocular positioning method. The slope displacement acquisition module is used to acquire slope displacement based on the three-dimensional spatial coordinate information of the monitoring target point and the initial coordinate information of the monitoring target point. The step of performing distortion correction processing on the enhanced image according to the correction parameters to obtain the monitoring image includes: in, , , The three-dimensional spatial coordinates of the target point, , The pixel coordinates of the target point on the image plane.

Images