Slope retaining wall deformation monitoring method based on non-coplanar reference and multi-view reconstruction

Abstract

The application provides a kind of based on non-coplanar reference and multi-view reconstruction's slope retaining wall deformation monitoring method, it is related to high slope surface displacement monitoring technical field.The method includes: collecting the potential mutation site of slope retaining wall monitoring area multi-view image sequence, the image coordinates of each target center point are extracted after distortion correction to each multi-view image;Based on the image coordinates of the center point of non-coplanar reference point target, the world three-dimensional coordinates of non-coplanar reference point target, the exterior orientation element of each multi-view image is solved;Based on the exterior orientation element of each multi-view image, multi-view image sequence is carried out digital differential correction to generate the digital orthographic image of slope retaining wall monitoring area;Based on digital orthographic image, the three-dimensional displacement vector of monitoring point target relative to non-coplanar reference point is solved;Based on the vector, the deformation state of potential mutation site is judged.The application can realize the effective elimination of retaining wall perspective projection distortion error and single-view observation blind area error.

Description

Technical Field

[0001] This invention relates to the field of high slope surface displacement monitoring technology, and in particular to a slope retaining wall deformation monitoring method based on non-coplanar datum and multi-view reconstruction. Background Technology

[0002] As a key engineering structure for maintaining geological stability and preventing landslides, the deformation evolution of slope retaining walls directly affects the long-term reliability of the overall project and the safety of life and property in the surrounding area. Under the continuous action of reservoir water level fluctuations, rainfall, crustal micro-vibrations, and surrounding loads, slope retaining walls will undergo slow plastic deformation and may develop sudden sliding fracture surfaces.

[0003] Existing methods for monitoring slope and retaining wall deformation mainly fall into two categories: contact-based physical sensor monitoring and ground-based optical instrument monitoring. Contact sensors, such as linear variable differential transformers (LVDTs), strain gauges, and fiber optic grating sensors, can provide high-resolution local strain data, but their deployment in large-scale, rugged, and high-risk areas is difficult. Furthermore, power supply and cables are susceptible to damage from harsh outdoor environments, resulting in high maintenance costs. Ground-based optical instruments, such as total stations and ground-based interferometric synthetic aperture radar (GB-InSAR), can achieve non-contact measurements, but require strict line-of-sight between the observation station and each monitored target, which is difficult to meet in densely vegetated and complex terrain slope environments. In addition, the installation and point-by-point scanning of fixed equipment are time-consuming, making it difficult to implement high-frequency, full-coverage automated monitoring of slope groups stretching for tens of kilometers.

[0004] In recent years, with the maturity of unmanned aerial vehicle (UAV) automated airport technology, it has become possible to use UAVs equipped with cameras for high-frequency, unmanned slope inspections, providing a non-contact, wide-coverage solution for large-area deformation surveys. However, when applying UAV photogrammetry technology to slope deformation monitoring with millimeter-level precision, existing technologies still face the following insurmountable technical bottlenecks: First, the problem of false displacement caused by pose disturbances of the drone platform.

[0005] When a drone hovers and takes pictures, the camera's 3D position and 3D attitude in the world coordinate system will randomly change due to wind load, rotor airflow, and motor vibration. This introduces false pixel displacements on the image plane that often far exceed the millimeter-level actual deformation of the retaining wall itself. Existing technologies, such as patent CN122041745A, propose using a dual-reference point correction method to correct the data. However, from the perspective of photogrammetry and projective geometry model analysis, methods relying on dual reference points or homography matrices suffer from a coplanarity constraint. The derivation of homography transformation strictly depends on the assumption that all world coordinate points involved in the mapping lie on the same rigid plane. In actual engineering, the surface of the retaining wall is uneven, and reference point targets are often placed on higher ground or anti-slip piles behind the retaining wall to avoid being affected by landslides. These targets have a significant elevation difference in the depth direction from the retaining wall monitoring points and are not coplanar. Applying homography transformation calculated based on planar control points to non-coplanar regions with large depth differences fails to truly decouple the camera's six degrees of freedom perturbation. Instead, it generates image distortion and misalignment in the depth difference region, misjudging the UAV's attitude deviation as deformation data, resulting in a massive number of false alarms.

[0006] Second, it is difficult to distinguish between real displacement and perspective scaling artifacts from a single perspective.

[0007] Current drone-based slope inspection methods mostly employ single-view scanning parallel to the retaining wall, relying on image optical flow or sub-pixel feature comparison to calculate two-dimensional apparent displacement. However, single-view two-dimensional offset cannot distinguish between the actual three-dimensional slippage of the retaining wall and the perspective magnification effect caused by changes in the distance between the drone and the retaining wall (e.g., a 5-centimeter reduction in distance). Although existing technologies such as patents CN115638726A and CN107843204A use multi-view intersection to extract depth, they mostly rely on fixed gimbals or camera arrays, making them difficult to adapt to large-area, unpowered outdoor slope environments, and failing to fully utilize the maneuverability advantages of drones.

[0008] Third, the problem of perspective projection eccentricity error during oblique photography.

[0009] When a drone takes multi-view verification photos around a monitoring point, it will inevitably image the circular target on the retaining wall from a large tilt angle (e.g., 45 to 70 degrees). In oblique perspective projection, the spatial circle is projected as an ellipse on the image plane, and there is a systematic deviation between the projected position of its geometric center and the real center of the circle, namely, perspective projection eccentricity error. In close-range photogrammetry or large tilt angle observations, this eccentricity error can reach several pixels. When the system ground sampling distance (GSD) is 1 mm, the eccentricity error itself can introduce a measurement deviation of 2 to 5 mm, exceeding the tolerance limit allowed by national standards.

[0010] In summary, existing technologies struggle to balance large-scale, high-frequency coarse-grained inspection with millimeter-level precision 3D deformation recognition in the context of UAV maneuvering and cruising environments. Summary of the Invention

[0011] This invention provides a slope retaining wall deformation monitoring method based on non-coplanar datum and multi-view reconstruction to solve the problems in slope retaining wall deformation monitoring, such as false displacement caused by platform pose disturbance, difficulty in removing perspective scaling artifacts in single-view observation, and projection eccentricity error introduced by large-angle shooting, which lead to the inability to accurately obtain the true three-dimensional deformation of the retaining wall.

[0012] In a first aspect, embodiments of the present invention provide a method for monitoring the deformation of a slope retaining wall based on a non-coplanar datum and multi-view reconstruction, comprising: Collect multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall; each frame of the multi-view image sequence simultaneously contains non-coplanar reference point targets and monitoring point targets corresponding to potential abrupt changes. Distortion correction is performed on each frame of the multi-view image sequence, and the image coordinates of each target center point after correction are extracted from each frame of the multi-view image. Based on the image coordinates of the center points of non-coplanar reference point targets and the world 3D coordinates of non-coplanar reference point targets in the image coordinates of each target center point after correction, the exterior orientation elements of each frame of multi-view image are calculated. Based on the exterior orientation elements of each frame of multi-view images, digital differential correction is performed on the multi-view image sequence to generate a digital orthophoto map of the slope retaining wall monitoring area. The three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point is calculated based on the digital orthophoto image interpretation. Deformation state of potential abrupt changes is determined based on three-dimensional displacement vectors.

[0013] In one possible implementation, prior to acquiring a multi-view image sequence of potential abrupt changes in the slope retaining wall monitoring area, the following is also included: Non-coplanar reference point targets and monitoring point targets are manually deployed in the slope retaining wall monitoring area; the non-coplanar reference point targets and monitoring point targets are located at different positions; Collect images of the monitoring area of ​​the slope retaining wall; Based on image extraction, the apparent pixel displacement of the target at each monitoring point is extracted; When the apparent pixel displacement of a target at a certain monitoring point exceeds a preset safety threshold, the area where the target at that monitoring point is located is recorded as a potential mutation site.

[0014] In one possible implementation, non-coplanar reference point targets and monitoring point targets are manually deployed in the slope retaining wall monitoring area, including: Multiple reference point targets that are not in the same plane are set up on the stable bedrock or immovable body around the slope retaining wall. Monitoring targets were set up in the deformation-sensitive areas of the slope retaining wall.

[0015] In one possible implementation, distortion correction is performed on each frame of the multi-view image sequence, and the image coordinates of each target center point after correction are extracted from each frame of the multi-view image, including: Convolution is performed on the local window image of the concentric target in each frame of multi-view image to extract sub-pixel edge points with continuous gray-level step features; The equations of the inner and outer ellipses are fitted based on the sub-pixel edge points of each frame of multi-view images to obtain the geometric center image coordinates of the inner ellipse and the geometric center image coordinates of the outer ellipse. Based on the geometric center image coordinates of the inner ellipse, the geometric center image coordinates of the outer ellipse, and the actual physical radius of the concentric circles, nonlinear eccentricity compensation is performed to obtain the image coordinates of the center points of each target after correction of each frame of multi-view images.

[0016] In one possible implementation, based on the image coordinates of the center points of non-coplanar reference point targets and the world 3D coordinates of the non-coplanar reference point targets in the image coordinates of each target center point after correction, the exterior orientation elements of each frame of multi-view image are calculated, including: Obtain the image coordinates of the corrected center points of multiple non-coplanar reference point targets distributed at different 3D depths in each frame of multi-view image, and construct the collinearity equation by combining the pre-established world 3D coordinates of the non-coplanar reference point targets. Based on the collinearity equation, an objective function is constructed with the goal of minimizing the reprojection error; The objective function is solved iteratively to obtain the exterior orientation rotation matrix and the exterior orientation translation vector; The exterior orientation rotation matrix and exterior orientation translation vector are used as the exterior orientation elements of each frame of multi-view image.

[0017] In one possible implementation, digital differential correction is performed on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view images to generate a digital orthophoto map of the slope retaining wall monitoring area, including: Using the global world coordinate system constructed with non-coplanar reference point targets as a reference, an object projection reference surface is constructed for the slope retaining wall monitoring area; Based on the exterior orientation elements of each frame of multi-view images and the interior orientation elements of the camera, establish the collinearity condition equation between the object points on the object projection reference plane and the corresponding image points on the original image. The orthophoto grid points on the object-side projection reference plane are traversed. The collinearity condition equation is used to calculate the corresponding image point coordinates of each orthophoto grid point on the original image. The corresponding gray values ​​are obtained through grayscale resampling to generate a digital orthophoto map.

[0018] In one possible implementation, the three-dimensional displacement vector of the monitoring point target relative to a non-coplanar reference point is calculated based on digital orthophoto image interpretation, including: Extract the two-dimensional orthorectified physical coordinates of the target at the monitoring point from the digital orthorectified image; Substituting the two-dimensional orthophoto physical coordinates into the three-dimensional geometric equation of the object-side projection reference plane, the current three-dimensional coordinates of the target at the monitoring point are obtained by solving the equation. The difference between the current three-dimensional coordinates and the reference three-dimensional coordinates is used to obtain the three-dimensional displacement vector of the target at the monitoring point.

[0019] In one possible implementation, the deformation state of potential abrupt changes is determined based on a three-dimensional displacement vector, including: Compare the three-dimensional displacement vector with the apparent pixel displacement; When the three-dimensional displacement vector is in the preset noise range and the apparent pixel displacement exceeds the preset warning threshold, it is determined that there is a perspective magnification artifact in the potential abrupt change area. When the three-dimensional displacement vector exceeds the preset deformation threshold, it is determined that there is actual deformation at the potential abrupt change location.

[0020] Secondly, embodiments of the present invention provide a slope retaining wall deformation monitoring device based on non-coplanar datum and multi-view reconstruction, comprising: The multi-view image sequence acquisition module is used to acquire multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall; each frame of the multi-view image sequence simultaneously contains non-coplanar reference point targets and monitoring point targets corresponding to potential abrupt changes. The distortion correction module is used to correct the distortion of each frame of the multi-view image sequence and extract the image coordinates of the center point of each target after correction in each frame of the multi-view image. The solution module is used to solve the exterior orientation elements of each frame of multi-view image based on the image coordinates of the center points of non-coplanar reference point targets and the world 3D coordinates of non-coplanar reference point targets in the image coordinates of each target center point after correction. The solution module is also used to perform digital differential correction on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view images to generate a digital orthophoto map of the slope retaining wall monitoring area. The solution module is also used to calculate the three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point based on the digital orthophoto image. The judgment module is used to determine the deformation state of potential abrupt changes based on the three-dimensional displacement vector.

[0021] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or any possible implementation thereof.

[0022] In this embodiment of the invention, by collecting multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall, it is possible to obtain multi-dimensional observation data of the same target under different spatial positions and shooting angles, thereby breaking the geometric limitations of a single orthophoto perspective two-dimensional projection, and thus providing a rich data source for subsequent calculation of real three-dimensional deformation using multi-view geometric constraints and effectively removing perspective scaling artifacts caused by changes in shooting distance or angle.

[0023] By performing distortion correction on each frame of the multi-view image sequence to obtain the corrected center image coordinates, the geometric center offset caused by the projection of a circular target as an ellipse during large-angle oblique photography can be corrected. This eliminates systematic perspective projection eccentricity error from the data source, thereby ensuring that the image coordinates input to the subsequent solution model have high fidelity and avoiding measurement deviations at the millimeter level.

[0024] By calculating the exterior orientation elements of each frame of multi-view images based on the corrected image coordinates and world 3D coordinates of the non-coplanar reference point target, the six-degree-of-freedom pose disturbance at the moment of UAV shooting can be accurately decoupled and compensated to eliminate false displacement. Then, based on the high-precision exterior orientation elements, the 3D displacement vector of the monitoring point target can be calculated, which can give the system depth perception capability to accurately restore the real physical slip of the retaining wall. Finally, it can effectively identify and eliminate perspective magnification / reduction artifacts caused by changes in shooting distance, ensuring the absolute authenticity of deformation data. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the implementation of a slope retaining wall deformation monitoring method based on non-coplanar datum and multi-view reconstruction provided by an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a slope retaining wall deformation monitoring device based on non-coplanar datum and multi-view reconstruction provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0026] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0027] See Figure 1 The document illustrates a flowchart of a slope retaining wall deformation monitoring method based on non-coplanar datum and multi-view reconstruction provided by an embodiment of the present invention, detailed below: Step 101: Collect multi-view image sequences of potential abrupt changes in the slope retaining wall monitoring area; each frame of the multi-view image sequence contains both non-coplanar reference point targets and monitoring point targets corresponding to potential abrupt changes.

[0028] In this step, after receiving the spatial coordinates of the potential mutation site, the UAV adjusts its flight trajectory and performs non-coplanar, multi-view hovering image acquisition around the suspected site, including but not limited to at least three differentiated tilt angles such as overhead, left tilt at 45 degrees, and right tilt at 45 degrees. The acquired multi-view image sequence simultaneously includes the target area and non-coplanar reference point targets distributed at different elevations within the field of view, and the image overlap between each viewpoint is no less than 70%. The multi-view image sequence acquired in this step has a strict constraint: each frame of the multi-view image sequence must simultaneously include both the non-coplanar reference point targets and the monitoring point targets corresponding to the potential mutation site.

[0029] By acquiring multi-view images of identified potential mutation sites, multi-dimensional observation data of the same target at different spatial locations and shooting angles can be obtained. The acquisition of multi-view images can overcome the geometric limitations of a single orthophoto perspective two-dimensional projection, providing a rich data source for subsequent calculation of true three-dimensional deformation using multi-view geometric constraints and effectively removing perspective scaling artifacts caused by changes in shooting distance or angle.

[0030] Step 102: Perform distortion correction on each frame of the multi-view image sequence and extract the image coordinates of the target center points after correction for each frame of the multi-view image.

[0031] In this step, by performing distortion correction on each frame of multi-view images, the geometric center offset caused by the projection of a circular target as an ellipse during large-angle oblique photography can be corrected. This eliminates systematic perspective projection eccentricity error from the data source, ensuring that the image coordinates input to the subsequent solution model have high fidelity and avoiding measurement deviations at the millimeter level.

[0032] Step 103: Based on the image coordinates of the center points of non-coplanar reference point targets and the world 3D coordinates of non-coplanar reference point targets in the image coordinates of each target center point after correction, calculate the exterior orientation elements of each frame of multi-view image.

[0033] In this step, based on the corrected target image coordinates output in step 102, the center point image coordinates of the non-coplanar reference point targets are selected, and the solution is completed by combining them with the pre-measured and fixed world 3D coordinates of the non-coplanar reference point targets. By solving the exterior orientation elements of each frame of multi-view images using the above two sets of coordinates, the six-degree-of-freedom pose disturbances of the UAV at the moment of shooting can be accurately decoupled and compensated to eliminate false displacements. Since the reference points have a significant elevation difference in the Z-axis direction, the collinearity equation can mathematically decouple the UAV's 3D translation and 3D rotation independently, thereby breaking the coplanar limitation of the existing dual-reference / homography method and effectively eliminating the high-frequency disturbances caused by wind loads on the UAV fuselage.

[0034] Step 104: Based on the exterior orientation elements of each frame of multi-view images, perform digital differential correction on the multi-view image sequence to generate a digital orthophoto map of the slope retaining wall monitoring area.

[0035] In this step, based on the camera exterior orientation elements obtained frame by frame in step 103, digital differential correction processing is carried out on the entire set of multi-view image sequences after distortion correction to generate a digital orthophoto map that fully covers the monitoring area of ​​the slope retaining wall. The orthophoto map eliminates the projection distortion caused by different shooting angles and shooting distances, unifies the imaging benchmark of all monitoring targets, and provides a standardized two-dimensional image carrier for subsequent three-dimensional displacement calculation.

[0036] Step 105: Calculate the three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point based on the digital orthophoto image.

[0037] In this step, the digital orthophoto image generated in step 104 is used as the computational basis to calculate the three-dimensional displacement vectors of all monitoring point targets within the image relative to non-coplanar reference points. By calculating the three-dimensional displacement vectors of the monitoring point targets based on exterior orientation elements, the system is endowed with depth perception capabilities to accurately reproduce the true physical slippage of the retaining wall. The multi-view spatial forward intersection theory can calculate the true three-dimensional coordinates of the monitoring points from the intersection of light rays from multiple perspectives, thereby obtaining accurate three-dimensional displacement vectors.

[0038] Step 106: Determine the deformation state of potential abrupt change sites based on the three-dimensional displacement vector.

[0039] In this step, by comparing the three-dimensional displacement vector with the apparent pixel displacement, perspective magnification artifacts caused by changes in shooting distance can be effectively identified and eliminated, ensuring the absolute authenticity of deformation data and reducing the cost of ineffective emergency responses. Simultaneously, for confirmed genuine deformations, the system can output precise three-dimensional deformation vectors, providing spatial guidance for the targeted deployment of on-site contact precision instruments.

[0040] In one possible implementation, prior to acquiring a multi-view image sequence of potential abrupt changes in the slope retaining wall monitoring area, the following is also included: Non-coplanar reference point targets and monitoring point targets are manually deployed in the slope retaining wall monitoring area; the non-coplanar reference point targets and monitoring point targets are located at different positions; Collect images of the monitoring area of ​​the slope retaining wall; Based on image extraction, the apparent pixel displacement of the target at each monitoring point is extracted; When the apparent pixel displacement of a target at a certain monitoring point exceeds a preset safety threshold, the area where the target at that monitoring point is located is recorded as a potential mutation site.

[0041] In this embodiment, by strategically deploying two types of targets with distinct locations within the monitoring area, a stable spatial reference benchmark is provided for subsequent acquisition of multi-view images containing both types of targets in each frame and for subsequent image processing. Non-coplanar reference point targets provide absolute coordinate references to eliminate the influence of UAV pose disturbances; monitoring point targets reflect the actual deformation of the retaining wall surface. This non-coplanar deployment effectively overcomes the limitations of traditional coplanar assumptions in the calculation process, improving the stability of exterior orientation element calculations.

[0042] Before acquiring multi-view image sequences, the drone is automatically released from the hangar according to a preset cycle and performs large-scale image acquisition on the retaining wall surface according to a preset orthophoto or single oblique angle flight path.

[0043] The acquired image sequences are transmitted back to the edge computing node, where sub-pixel offset tracking (sPOT) is used to perform a two-dimensional cross-correlation comparison between the monitoring point target and the natural texture of the retaining wall to extract the apparent pixel displacement. When the apparent pixel displacement of a monitoring point target exceeds a preset safety threshold, the area where the monitoring point target is located is recorded as a potential mutation site.

[0044] In one possible implementation, non-coplanar reference point targets and monitoring point targets are manually deployed in the slope retaining wall monitoring area, including: Multiple reference point targets that are not in the same plane are set up on the stable bedrock or immovable body around the slope retaining wall. Monitoring targets were set up in the deformation-sensitive areas of the slope retaining wall.

[0045] In this embodiment, to overcome the coplanarity assumption relied upon by the traditional dual-benchmark correction method, a non-coplanar benchmark target network is adopted. Specifically, multiple benchmark targets that are not in the same plane are deployed on the stable bedrock or immovable body around the slope retaining wall, such as exposed bedrock at the elevation behind the slope or the top of anti-slip piles deeply embedded below the sliding surface. The benchmark targets have a significant elevation span in three-dimensional space, with an elevation difference of no less than 5 meters between them in the direction perpendicular to the retaining wall (Z-axis), and are dispersed in the horizontal direction, thus forming a non-coplanar benchmark network. A high-precision total station is used to perform a one-time precise three-dimensional coordinate measurement of the benchmark points to establish a world absolute coordinate system, and the algorithm assumes that its true displacement is always zero. At the same time, monitoring target points are deployed on the monitoring surfaces of the slope retaining wall, such as deformation joints, settlement observation zones, both sides of structural cracks, and key stress nodes of the support structure.

[0046] Preferably, both the reference point target and the monitoring point target adopt a concentric circle pattern design, consisting of an inner circle with a known physical radius R1 and an outer circle with a known physical radius R2. The ratio of R2 / R1 ranges from 1.5 to 3.0, which is used for perspective eccentricity error compensation in subsequent steps.

[0047] In one possible implementation, distortion correction is performed on each frame of the multi-view image sequence, and the image coordinates of each target center point after correction are extracted from each frame of the multi-view image, including: Convolution is performed on the local window image of the concentric target in each frame of multi-view image to extract sub-pixel edge points with continuous gray-level step features; The equations of the inner and outer ellipses are fitted based on the sub-pixel edge points of each frame of multi-view images to obtain the geometric center image coordinates of the inner ellipse and the geometric center image coordinates of the outer ellipse. Based on the geometric center image coordinates of the inner ellipse, the geometric center image coordinates of the outer ellipse, and the actual physical radius of the concentric circles, nonlinear eccentricity compensation is performed to obtain the image coordinates of the center points of each target after correction of each frame of multi-view images.

[0048] In this embodiment, the local window image of the concentric circular target is first convolved using the pseudo-Zernike moment orthogonal polynomial operator to extract sub-pixel edge points with continuous gray-level step features. Then, RANSAC robust estimation combined with least squares method is used to fit the equations of the inner and outer ellipses to obtain the geometric center image coordinates of the inner ellipse. Image coordinates of the geometric center point of the outer ellipse .

[0049] Then, based on the image coordinates of the geometric center point of the inner ellipse... Image coordinates of the geometric center point of the external ellipse Nonlinear eccentricity compensation is performed on the actual physical radii R1 and R2 of the concentric circles.

[0050] Finally, based on the geometric characteristics of concentric circular projection, a nonlinear eccentricity compensation analytical model is applied to approximate the true perspective projection center, thereby obtaining the corrected center image coordinates for each frame of multi-view images. This model utilizes only the geometric relationship between the inner and outer circles of the concentric circles, without requiring prior information on the camera's exterior orientation, to algebraically approximate the true projection center on the image plane, compressing the perspective eccentricity error at large tilt angles from several pixels to the sub-pixel level.

[0051] The nonlinear eccentricity compensation analytical model is as follows: In the formula, R1 and R2 are the actual physical radii of the inner and outer circles of the concentric circles, respectively.

[0052] In one possible implementation, based on the image coordinates of the center points of non-coplanar reference point targets and the world 3D coordinates of the non-coplanar reference point targets in the image coordinates of each target center point after correction, the exterior orientation elements of each frame of multi-view image are calculated, including: Obtain the corrected center image coordinates of multiple non-coplanar reference point targets distributed at different 3D depths in each frame of multi-view image, and construct the collinearity equation by combining the pre-established world 3D coordinates of the non-coplanar reference point targets. Based on the collinearity equation, an objective function is constructed with the goal of minimizing the reprojection error; The objective function is solved iteratively to obtain the exterior orientation rotation matrix and the exterior orientation translation vector; The exterior orientation rotation matrix and exterior orientation translation vector are used as the exterior orientation elements of each frame of multi-view image.

[0053] In this embodiment, for each frame of multi-view image that has undergone eccentricity compensation and simultaneously contains non-coplanar reference targets and monitoring targets, at least four non-coplanar reference point targets distributed at different three-dimensional depths within the field of view are identified. Their corrected center image coordinates are extracted, and combined with the pre-determined and fixed world three-dimensional coordinates of the reference points using a total station, a collinearity equation, namely the Perspective-n-Point (PnP) equation, is constructed: In the formula, For the camera intrinsic parameter matrix, Given the known three-dimensional coordinates of the reference point, For the corresponding image coordinates, and Let be the external orientation rotation matrix and translation vector to be determined, and s be the scale factor.

[0054] The rotation matrix R and translation vector t are iteratively solved using the Levenberg-Marquardt nonlinear least squares algorithm to obtain the initial exterior orientation elements of a single frame and separate the six-degree-of-freedom attitude perturbation of the UAV.

[0055] The exterior orientation rotation matrix and exterior orientation translation vector are used as the exterior orientation elements of each frame of multi-view image, which represents the six-degree-of-freedom pose of the UAV at the moment of capture in that frame. Since the reference points have a significant elevation difference in the Z-axis direction, the collinearity equation can mathematically decouple the UAV's three-dimensional translation and three-dimensional rotation, thereby breaking the coplanar limitation of the existing dual-reference / homography method and effectively eliminating the high-frequency disturbances caused by wind loads on the UAV fuselage.

[0056] Preferably, to further improve global accuracy, the independent initial poses of each frame can be placed in the time series of multi-view images to establish a global reprojection error objective function that includes camera pose parameters and the three-dimensional coordinates of monitoring points. A pre-deployed three-dimensional scale constraint, such as a carbon fiber scale of known physical length or the absolute distance between reference point pairs, is injected as a high-weight penalty function term. Global optimization is performed through joint bundle adjustment to obtain an absolute coordinate system that is static in time and has a unified scale in space.

[0057] The objective function for global reprojection error is as follows: In the formula, the first term is the reprojection error. For projection function, Here, the second term is the Huber robust kernel function, and the third term is the penalty function term for the scaling constraint. For known physical lengths (length of carbon fiber ruler or absolute distance between reference point pairs). The weighting coefficients of the penalty function have a range of 10. 4 Up to 10 6 .

[0058] In one possible implementation, digital differential correction is performed on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view images to generate a digital orthophoto map of the slope retaining wall monitoring area, including: Using the global world coordinate system constructed with non-coplanar reference point targets as a reference, an object projection reference surface is constructed for the slope retaining wall monitoring area; Based on the exterior orientation elements of each frame of multi-view images and the interior orientation elements of the camera, establish the collinearity condition equation between the object points on the object projection reference plane and the corresponding image points on the original image. The orthophoto grid points on the object-side projection reference plane are traversed. The collinearity condition equation is used to calculate the corresponding image point coordinates of each orthophoto grid point on the original image. The corresponding gray values ​​are obtained through grayscale resampling to generate a digital orthophoto map.

[0059] In this embodiment, digital differential correction is performed on the entire set of multi-view image sequences. A global world coordinate system based on a non-coplanar reference network is established to create a reference surface for the object projection of the slope retaining wall. The mapping relationship between the object points and the original image points is established by combining the internal and external orientation elements of the camera. All orthorectified grids on the reference surface are traversed one by one, and the corresponding image coordinates of the grid points in the original image are solved in reverse. Grayscale resampling is used to fill the pixel grayscale, and finally a digital orthorectified image of the slope retaining wall monitoring area with perspective distortion eliminated and a unified coordinate reference is generated, providing a standardized image carrier for subsequent three-dimensional displacement calculation.

[0060] In one possible implementation, the three-dimensional displacement vector of the monitoring point target relative to a non-coplanar reference point is calculated based on digital orthophoto image interpretation, including: Extract the two-dimensional orthorectified physical coordinates of the target at the monitoring point from the digital orthorectified image; Substituting the two-dimensional orthophoto physical coordinates into the three-dimensional geometric equation of the object-side projection reference plane, the current three-dimensional coordinates of the target at the monitoring point are obtained by solving the equation. The difference between the current three-dimensional coordinates and the reference three-dimensional coordinates is used to obtain the three-dimensional displacement vector of the target at the monitoring point.

[0061] In this embodiment, based on the theory of spatial triangulation, spatial projection rays are constructed from the optical centers of each camera through the image points of the monitoring target in the corresponding images, according to the exterior orientation elements. A intersection error equation is constructed with the goal of minimizing the spatial distance between the spatial projection rays. The intersection error equation is solved using the least squares method to obtain the current three-dimensional coordinates of the monitoring target. The difference between the current three-dimensional coordinates and the reference three-dimensional coordinates is used to obtain the three-dimensional displacement vector of the monitoring target.

[0062] In one possible implementation, the deformation state of potential abrupt changes is determined based on a three-dimensional displacement vector, including: Compare the three-dimensional displacement vector with the apparent pixel displacement; When the three-dimensional displacement vector is in the preset noise range and the apparent pixel displacement exceeds the preset warning threshold, it is determined that there is a perspective magnification artifact in the potential abrupt change area. When the three-dimensional displacement vector exceeds the preset deformation threshold, it is determined that there is actual deformation at the potential abrupt change location.

[0063] In this embodiment, the three-dimensional displacement vector is logically compared with the acquired apparent pixel displacement.

[0064] When the three-dimensional displacement vector is within a preset noise range (e.g., the sub-millimeter noise range, typically ≤0.5 mm) and the apparent pixel displacement exceeds a preset warning threshold, it is determined that there is a perspective magnification artifact at the potential abrupt change location. That is, the preliminary result is a perspective magnification effect caused by a brief change in the distance between the drone and the retaining wall. This artifact is removed from the warning queue to avoid ineffective emergency response.

[0065] When the three-dimensional displacement vector exceeds the preset deformation threshold, it is determined that there is actual deformation at the potential abrupt change location. At this time, the system generates a deformation monitoring report containing three-dimensional geographic coordinates, displacement vectors, and a visualization model, and pushes it to the slope monitoring and dispatch center to guide on-site engineers to carry contact precision instruments to the designated location to carry out targeted and close physical monitoring.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0067] The following are device embodiments of the present invention. For details not described in detail, please refer to the corresponding method embodiments described above.

[0068] Figure 2 The diagram shows a structural schematic of a slope retaining wall deformation monitoring device based on non-coplanar datum and multi-view reconstruction according to an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, and are described in detail below: like Figure 2 As shown, the slope retaining wall deformation monitoring device 2 based on non-coplanar datum and multi-view reconstruction includes: The multi-view image sequence acquisition module 201 is used to acquire multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall; each frame of the multi-view image sequence contains both non-coplanar reference point targets and monitoring point targets corresponding to potential abrupt changes. The distortion correction module 202 is used to perform distortion correction on each frame of the multi-view image sequence and extract the image coordinates of each target center point after correction in each frame of the multi-view image. The solution module 203 is used to solve the exterior orientation elements of each frame of multi-view image based on the image coordinates of the center point of the non-coplanar reference point target and the world three-dimensional coordinates of the non-coplanar reference point target in the image coordinates of each target center point after correction. The solution module 203 is also used to perform digital differential correction on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view image to generate a digital orthophoto map of the slope retaining wall monitoring area. The calculation module 203 is also used to calculate the three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point based on the digital orthophoto image. The judgment module 204 is used to judge the deformation state of potential abrupt changes based on the three-dimensional displacement vector.

[0069] In one possible implementation, the slope retaining wall deformation monitoring device 2 based on non-coplanar datum and multi-view reconstruction further includes a sudden change location determination module 205, which is used for: Non-coplanar reference point targets and monitoring point targets are manually deployed in the slope retaining wall monitoring area; the non-coplanar reference point targets and monitoring point targets are located at different positions; Collect images of the monitoring area of ​​the slope retaining wall; Based on image extraction, the apparent pixel displacement of the target at each monitoring point is extracted; When the apparent pixel displacement of a target at a certain monitoring point exceeds a preset safety threshold, the area where the target at that monitoring point is located is recorded as a potential mutation site.

[0070] In one possible implementation, the mutation site determination module 205 is further configured to: Multiple reference point targets that are not in the same plane are set up on the stable bedrock or immovable body around the slope retaining wall. Monitoring targets were set up in the deformation-sensitive areas of the slope retaining wall.

[0071] In one possible implementation, the distortion correction module 202 is further used for: Convolution is performed on the local window image of the concentric target in each frame of multi-view image to extract sub-pixel edge points with continuous gray-level step features; The equations of the inner and outer ellipses are fitted based on the sub-pixel edge points of each frame of multi-view images to obtain the geometric center image coordinates of the inner ellipse and the geometric center image coordinates of the outer ellipse. Based on the geometric center image coordinates of the inner ellipse, the geometric center image coordinates of the outer ellipse, and the actual physical radius of the concentric circles, nonlinear eccentricity compensation is performed to obtain the image coordinates of the center points of each target after correction of each frame of multi-view images.

[0072] In one possible implementation, the solution module 203 is also used for: Obtain the image coordinates of the corrected center points of multiple non-coplanar reference point targets distributed at different 3D depths in each frame of multi-view image, and construct the collinearity equation by combining the pre-established world 3D coordinates of the non-coplanar reference point targets. Based on the collinearity equation, an objective function is constructed with the goal of minimizing the reprojection error; The objective function is solved iteratively to obtain the exterior orientation rotation matrix and the exterior orientation translation vector; The exterior orientation rotation matrix and exterior orientation translation vector are used as the exterior orientation elements of each frame of multi-view image.

[0073] In one possible implementation, the solution module 203 is also used for: Extract the two-dimensional orthorectified physical coordinates of the target at the monitoring point from the digital orthorectified image; Substituting the two-dimensional orthophoto physical coordinates into the three-dimensional geometric equation of the object-side projection reference plane, the current three-dimensional coordinates of the target at the monitoring point are obtained by solving the equation. The difference between the current three-dimensional coordinates and the reference three-dimensional coordinates is used to obtain the three-dimensional displacement vector of the target at the monitoring point.

[0074] In one possible implementation, the solution module 203 is also used for: Extract the two-dimensional orthorectified physical coordinates of the target at the monitoring point from the digital orthorectified image; Substituting the two-dimensional orthophoto physical coordinates into the three-dimensional geometric equation of the object-side projection reference plane, the current three-dimensional coordinates of the target at the monitoring point are obtained by solving the equation. The difference between the current three-dimensional coordinates and the reference three-dimensional coordinates is used to obtain the three-dimensional displacement vector of the target at the monitoring point.

[0075] In one possible implementation, the judgment module 204 is used for: Compare the three-dimensional displacement vector with the apparent pixel displacement; When the three-dimensional displacement vector is in the preset noise range and the apparent pixel displacement exceeds the preset warning threshold, it is determined that there is a perspective magnification artifact in the potential abrupt change area. When the three-dimensional displacement vector exceeds the preset deformation threshold, it is determined that there is actual deformation at the potential abrupt change location.

[0076] In one possible implementation, the slope retaining wall deformation monitoring device 2 based on non-coplanar datum and multi-view reconstruction also includes a digital orthophoto image generation module 206, which generates images for: Using the global world coordinate system constructed with non-coplanar reference point targets as a reference, an object projection reference surface is constructed for the slope retaining wall monitoring area; Based on the exterior orientation elements of each frame of multi-view images and the interior orientation elements of the camera, establish the collinearity condition equation between the object points on the object projection reference plane and the corresponding image points on the original image. The orthophoto grid points on the object-side projection reference plane are traversed. The collinearity condition equation is used to calculate the corresponding image point coordinates of each orthophoto grid point on the original image. The corresponding gray values ​​are obtained through grayscale resampling to generate a digital orthophoto map.

[0077] Figure 3 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. For example... Figure 3As shown, the electronic device 3 of this embodiment includes a processor 30 and a memory 31. The memory 31 stores a computer program 32. When the processor 30 executes the computer program 32, it implements the steps in the various method embodiments described above. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the various device embodiments described above.

[0078] For example, computer program 32 may be divided into one or more modules / units, which are stored in memory 31 and executed by processor 30 to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in electronic device 3.

[0079] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 3 may also include input / output devices, network access devices, buses, etc.

[0080] The processor 30 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0081] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 3. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 is used to store the computer program 32 and other programs and data required by the electronic device 3. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0082] For the sake of simplicity and clarity, only the above-described functional modules / units are used as examples. In practical applications, the functions described above can be assigned to different functional modules / units as needed. These modules / units can be implemented in hardware, software, or a combination of both.

[0083] This invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0084] This invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the methods described in the above-described method embodiments.

[0085] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0086] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0087] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for monitoring the deformation of slope retaining walls based on non-coplanar datum and multi-view reconstruction, characterized in that, include: Collect multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall; Each frame of the multi-view image sequence simultaneously contains non-coplanar reference point targets and monitoring point targets corresponding to the potential mutation sites. Distortion correction is performed on each frame of the multi-view image sequence, and the image coordinates of each target center point after correction are extracted from each frame of the multi-view image. Based on the image coordinates of the center point of the non-coplanar reference point target and the world three-dimensional coordinates of the non-coplanar reference point target in the image coordinates of each target center point after correction, the exterior orientation elements of each frame of multi-view image are calculated. Based on the exterior orientation elements of each frame of multi-view images, digital differential correction is performed on the multi-view image sequence to generate a digital orthophoto map of the slope retaining wall monitoring area. The three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point is calculated based on the digital orthophoto image. The deformation state of the potential mutation site is determined based on the three-dimensional displacement vector.

2. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, Before acquiring the multi-view image sequence of potential abrupt changes in the slope retaining wall monitoring area, the following is also included: The non-coplanar reference point targets and the monitoring point targets are manually deployed in the slope retaining wall monitoring area; the non-coplanar reference point targets and the monitoring point targets are located at different positions; Images of the monitoring area of ​​the slope retaining wall were acquired; Based on the image, the apparent pixel displacement of the target at each monitoring point is extracted; When the apparent pixel displacement of a target at a certain monitoring point exceeds a preset safety threshold, the area where the target at that monitoring point is located is recorded as a potential mutation site.

3. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 2, characterized in that, The manual deployment of the non-coplanar reference point targets and the monitoring point targets in the slope retaining wall monitoring area includes: Multiple reference point targets that are not in the same plane are set up on the stable bedrock or immovable body around the slope retaining wall. Monitoring points and targets are set up in the deformation-sensitive area of ​​the slope retaining wall.

4. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, The step of performing distortion correction on each frame of the multi-view image sequence and extracting the image coordinates of each target center point after correction in each frame of the multi-view image sequence includes: Convolution is performed on the local window image of the concentric target in each frame of multi-view image to extract sub-pixel edge points with continuous gray-level step features; The equations of the inner and outer ellipses are fitted based on the sub-pixel edge points of each frame of multi-view images to obtain the geometric center image coordinates of the inner ellipse and the geometric center image coordinates of the outer ellipse. Based on the geometric center image coordinates of the inner ellipse, the geometric center image coordinates of the outer ellipse, and the actual physical radius of the concentric circles, nonlinear eccentricity compensation is performed to obtain the image coordinates of each target center point after correction of each frame of multi-view image.

5. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, The exterior orientation elements of each frame of multi-view image are calculated based on the image coordinates of the center points of the non-coplanar reference point targets and the world 3D coordinates of the non-coplanar reference point targets in the image coordinates of each target center point after correction, including: Obtain the image coordinates of the corrected center points of multiple non-coplanar reference point targets distributed at different three-dimensional depths in each frame of multi-view image, and construct collinearity equations by combining the pre-established world three-dimensional coordinates of the non-coplanar reference point targets. Based on the collinearity equation, an objective function is constructed with the goal of minimizing the reprojection error; The objective function is solved iteratively to obtain the exterior orientation rotation matrix and the exterior orientation translation vector; The exterior orientation rotation matrix and the exterior orientation translation vector are used as the exterior orientation elements of each frame of multi-view image.

6. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, The step of performing digital differential correction on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view images to generate a digital orthophoto map of the slope retaining wall monitoring area includes: Using the global world coordinate system constructed from the non-coplanar reference point targets as a reference, an object projection reference surface for the slope retaining wall monitoring area is constructed. Based on the exterior orientation elements of each frame of multi-view image and the interior orientation elements of the camera, establish the collinearity condition equation between the object point on the object projection reference plane and the corresponding image point on the original image. The orthophoto grid points on the object projection reference plane are traversed, and the collinearity condition equation is used to calculate the corresponding image point coordinates of each orthophoto grid point on the original image. The corresponding gray value is obtained through grayscale resampling to generate the digital orthophoto image.

7. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, The calculation of the three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point based on the digital orthophoto image includes: Extract the two-dimensional orthorectified physical coordinates of the target at the monitoring point from the digital orthophoto image; Substituting the two-dimensional orthophoto physical coordinates into the three-dimensional geometric equation of the object-side projection reference surface, the current three-dimensional coordinates of the monitoring point target are obtained by solving the equation. The three-dimensional displacement vector of the monitoring point target is obtained by subtracting the current three-dimensional coordinates from the reference three-dimensional coordinates.

8. The method for monitoring slope retaining wall deformation based on non-coplanar datum and multi-view reconstruction according to claim 1, characterized in that, The determination of the deformation state of the potential abrupt change site based on the three-dimensional displacement vector includes: The three-dimensional displacement vector is compared with the apparent pixel displacement; When the three-dimensional displacement vector is in a preset noise range and the apparent pixel displacement exceeds a preset warning threshold, it is determined that there is a perspective magnification artifact in the potential mutation site. When the three-dimensional displacement vector exceeds a preset deformation threshold, it is determined that the potential mutation site has actual deformation.

9. A slope retaining wall deformation monitoring device based on non-coplanar datum and multi-view reconstruction, characterized in that, include: The multi-view image sequence acquisition module is used to acquire multi-view image sequences of potential abrupt changes in the monitoring area of ​​the slope retaining wall; each frame of the multi-view image sequence simultaneously contains non-coplanar reference point targets and monitoring point targets corresponding to the potential abrupt changes. The distortion correction module is used to perform distortion correction on each frame of the multi-view image sequence and extract the image coordinates of each target center point after correction in each frame of the multi-view image. The calculation module is used to calculate the exterior orientation elements of each frame of multi-view image based on the image coordinates of the center point of the non-coplanar reference point target and the world three-dimensional coordinates of the non-coplanar reference point target in the image coordinates of each target center point after correction. The solution module is also used to perform digital differential correction on the multi-view image sequence based on the exterior orientation elements of each frame of multi-view image to generate a digital orthophoto map of the slope retaining wall monitoring area. The calculation module is also used to calculate the three-dimensional displacement vector of the monitoring point target relative to the non-coplanar reference point based on the digital orthophoto image. The judgment module is used to determine the deformation state of the potential abrupt change site based on the three-dimensional displacement vector.

10. An electronic device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1 to 6.

Images