A roadbed deformation monitoring method and system

CN118762302BActive Publication Date: 2024-12-06HUNAN COMM RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411253268.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-12-06
Estimated Expiration
2044-09-09

Smart Images

  • Figure CN118762302B_ABST
    Figure CN118762302B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of computer vision technology, and in particular to a method and system for monitoring roadbed deformation. The method comprises the following steps: using a drone to collect multi-view image data of a highway roadbed to obtain multi-view image data; extracting feature points based on the multi-view image data to obtain multi-view image feature point data; matching feature points based on the multi-view image feature point data to obtain feature point matching data; performing motion recovery structure processing based on the feature point matching data to obtain sparse point cloud data and camera pose data; performing multi-view stereo matching based on the sparse point cloud data and the camera pose data to obtain roadbed dense point cloud data; performing multi-scale point cloud comparison based on the roadbed dense point cloud data to obtain roadbed deformation data, so as to perform roadbed deformation monitoring operations. The present invention realizes real-time monitoring and precise analysis of roadbed deformation, and provides strong technical support for the maintenance and management of highways.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a roadbed deformation monitoring method and system. Background Art

[0002] Roadbed engineering is the fundamental part of highway construction. The stability of the roadbed is greatly affected by unfavorable factors such as poor construction quality, rainwater infiltration, and complex geological conditions. It is very easy to cause road cutting landslides, embankment subsidence and other diseases, so it is very necessary to monitor the deformation of the roadbed. This can not only analyze the causes of displacement and settlement changes, but also provide early warning of disasters in a timely manner, minimize the economic and social impacts caused by disasters, and help to gain a deeper understanding of the nature of the development of the roadbed stability state.

[0003] With the rapid development of new technologies, a variety of deformation monitoring technologies have been formed. These technologies have their own advantages, but also their own limitations. Traditional deformation monitoring technology usually selects some key points where disasters are prone to occur for monitoring, and uses this to infer the stability of the entire roadbed. However, in the face of a roadbed with a complex geological environment, due to the relatively scattered density of points, it is difficult to obtain the overall deformation of the roadbed through single-point monitoring. Ground-based three-dimensional laser scanning technology collects high-precision measurement data quickly through non-contact, but its cost is high and is greatly affected by environmental conditions, so it cannot be widely used. Satellite remote sensing imaging technology has low accuracy and is only suitable for large-scale disaster surveys. For roadbeds within a length of tens of meters, its accuracy is difficult to meet monitoring requirements, and it is difficult to ensure real-time performance. In short, the above methods have problems such as low efficiency, high risk, high cost, and low accuracy, and it is difficult to meet the efficiency and digitalization requirements of actual projects. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a roadbed deformation monitoring method and system to solve at least one of the above technical problems.

[0005] The present application provides a roadbed deformation monitoring method, the method comprising:

[0006] S1. Use a drone to collect multi-view image data of a highway subgrade to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; perform feature point matching based on the multi-view image feature point data to obtain feature point matching data;

[0007] S2. Perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data;

[0008] S3, performing multi-view stereo matching based on the sparse point cloud data and the camera pose data to obtain dense point cloud data of the roadbed;

[0009] S4. Perform multi-scale point cloud comparison based on the dense point cloud data of the roadbed to obtain the roadbed deformation data for roadbed deformation monitoring and deformation operation.

[0010] In the present invention, an unmanned aerial vehicle is used to collect multi-view image data of a highway roadbed, and the aerial images are preprocessed, feature points are extracted, and feature points are matched. A motion recovery structure algorithm is used to obtain a sparse point cloud of the roadbed and a camera pose. Then, a multi-view stereo matching algorithm is used to obtain dense point cloud data of the roadbed. Subsequently, changes in multi-period point cloud data are compared based on a multi-scale model and a model point cloud comparison algorithm M3C2. Finally, deformation monitoring of the highway roadbed is achieved by visually expressing changes in displacement and settlement of the roadbed.

[0011] Optionally, the multi-view image data includes first multi-view image data and second multi-view image data, and the multi-view image feature point data includes first multi-view image feature point data and second multi-view image feature point data, and S1 includes:

[0012] Using the drone to collect multi-view image data of the highway roadbed through a preset first threshold height data to obtain first multi-view image data;

[0013] Using the drone to collect multi-view image data of the highway roadbed through the preset second threshold height data to obtain second multi-view image data, wherein the first threshold height data is greater than the second threshold height data;

[0014] Performing a first feature point detection on the first multi-view image data to obtain the first multi-view image feature point data, and performing a second feature point detection on the second multi-view image data to obtain the second multi-view image feature point data, wherein the first feature point detection and the second feature point detection are different feature point detection methods;

[0015] Generate a descriptor based on the first multi-view image feature point data to obtain first descriptor data, and generate a descriptor based on the second multi-view image feature point data to obtain second descriptor data;

[0016] A matching calculation is performed based on the first descriptor data and the second descriptor data to obtain feature point matching data.

[0017] In the present invention, the first threshold height provides a wider field of view, which is suitable for capturing global structures; the second threshold height provides a more detailed close-range perspective, which is suitable for capturing details and microscopic changes. Combining different viewing angle data at different heights can more accurately restore the true form of the three-dimensional scene, especially more effectively capturing details of complex terrain. Using different detection methods (such as SIFT, ORB, etc.), different types of feature points can be captured. One method may be good at detecting large-scale structural features, while another may be more suitable for extracting detail features. By combining multiple methods, the problem of missing feature points that may occur in a single method can be reduced, and the robustness of the detection results can be improved. Generating different descriptors based on different feature point data helps to better adapt to various image features and enhance the ability to distinguish feature points. Different descriptors can provide diverse feature representations for the same scene, which helps to improve the accuracy of matching. By matching and calculating descriptors from different heights and different methods, it is possible to make full use of multiple feature information and improve the accuracy and reliability of matching.

[0018] Optionally, the first feature point detection is specifically:

[0019] Performing color correction on the first multi-view image data to obtain color-corrected data;

[0020] Performing multi-scale spatial construction on the color correction data to obtain multi-scale spatial construction data;

[0021] Key point detection is performed on the multi-scale space construction data to obtain first multi-view image feature point data.

[0022] The color correction in the present invention helps to eliminate color inconsistencies caused by different lighting conditions, camera parameters or atmospheric conditions, so that the colors of all images are more uniform and natural. By eliminating color differences, the stability of subsequent feature point detection can be enhanced, and feature point detection deviations caused by color differences can be avoided. Appropriate color correction can improve the contrast of the image, thereby more clearly highlighting the feature areas in the image. Multi-scale space construction can capture the features of the image at different scales, and can detect large-scale macro features as well as subtle local features. By analyzing images of different scales, the inaccuracy of feature point detection caused by scale changes can be reduced, and the robustness of feature points can be improved. Through key point detection, highly significant feature points can be extracted from multi-scale images, which is convenient for subsequent feature matching and three-dimensional reconstruction. Since key points are usually detected based on local information of the image (such as gradients, edges, etc.), the impact of noise on feature detection can be reduced.

[0023] Optionally, the multi-scale space construction data includes first multi-scale space construction data and second multi-scale space construction data, and the multi-scale space construction is specifically:

[0024] Performing texture multi-scale space construction on the color correction data to obtain first multi-scale space construction data;

[0025] Performing region division on the color correction data to obtain color correction region division data;

[0026] Perform color feature extraction based on the color correction area division data to obtain regional color feature data;

[0027] The color correction data is subjected to color multi-scale space construction according to the regional color feature data to obtain second multi-scale space construction data.

[0028] The present invention can analyze the texture pattern in the image at multiple scales, from the global texture structure to the local detail features, and provide a comprehensive texture description. Multi-scale analysis can distinguish texture changes at different scales, especially in areas with complex textures and rich details, and provide more accurate feature extraction. By dividing the region, the features in the specific region can be analyzed intensively to avoid dilution of the overall image features. Regional division can help focus on local color changes when extracting color features, rather than the global color features of the entire image. Through regional color feature extraction, the color distribution and changes in different regions of the image can be captured, which helps to understand the content and scene of the image. By constructing a color multi-scale space, the changes in color features in the image at different scales can be analyzed, and the color features at large and small scales can be identified. Multi-scale color features can help maintain the consistency and accuracy of color features at different resolutions, and help to improve the accuracy of feature matching.

[0029] Optionally, the texture multi-scale space construction is specifically as follows:

[0030] Performing Gaussian blur on the color correction data to obtain Gaussian blurred data;

[0031] Down-sample the Gaussian blurred data to obtain down-sampled data;

[0032] Extracting texture features from the downsampled data to obtain downsampled texture feature data;

[0033] Performing texture feature inversion on the downsampled data according to the downsampled texture feature data to obtain downsampled texture feature inversion data;

[0034] Performing up-sampling processing on the down-sampled texture feature inversion data to obtain up-sampled data;

[0035] Subtraction processing is performed on the up-sampled data and the down-sampled data to obtain first multi-scale space construction data.

[0036] In the present invention, the color-corrected image data is subjected to Gaussian blur processing to reduce noise and details and obtain Gaussian blurred data. Downsampling reduces the resolution of the image, thereby reducing the amount of data and computational complexity, and is suitable for multi-scale analysis. By constructing a low-resolution version, the global features of the image and a larger range of patterns can be analyzed. Extracting texture features helps to identify details and local patterns in the image, especially on low-resolution images, where these features are more prominent. Through the inversion operation, the part opposite to the original feature can be enhanced or highlighted, providing a different perspective to observe the image content. Upsampling restores part of the original resolution so that the data can be compared with the high-resolution features. In the process of restoring the resolution, the information of the inverted features is combined to further highlight or smooth certain details. Subtraction processing can highlight the edges and details in the image, similar to the effect of the Laplace pyramid, and strengthen the change information in the image. By analyzing the difference between the upsampled data and the original data, the changes and feature distribution in the image can be better understood.

[0037] Optionally, the second feature point detection is specifically:

[0038] Performing image enhancement on the second multi-view image data to obtain image enhancement data;

[0039] Perform corner point detection on the image enhancement data to obtain corner point detection data;

[0040] The feature point directions are calculated for the angle detection data to obtain the second multi-view image feature point data.

[0041] Image enhancement in the present invention can improve the contrast and sharpness of the image, making the features easier to detect. By enhancing the local contrast, important image features such as corners and edges can be presented more clearly, which is convenient for subsequent feature point detection. Corner point detection provides rich local information and plays an important role in the global analysis and matching of image features. Calculating the direction of feature points helps to achieve the directional invariance of feature points, so that these feature points can still be matched when the image is rotated or flipped. By assigning a direction information to each feature point, the direction information can be used in the matching process to improve the accuracy and reliability of the matching. The direction information adds a dimension of description to each feature point, which helps to more accurately locate and identify feature points.

[0042] Optionally, S2 includes:

[0043] Select image pairs based on feature point matching data to obtain initial image pair data;

[0044] The camera pose is estimated based on the initial image pair data and the preset drone camera internal parameter data to obtain the initial camera pose data;

[0045] Generate sparse point cloud according to initial image pair data and initial camera pose data to obtain preliminary sparse point cloud data;

[0046] Perform camera pose estimation and sparse point cloud generation based on adjacent image pair data in the feature point matching data corresponding to the initial image pair data, and obtain adjacent camera pose data and adjacent sparse point cloud data;

[0047] Performing bundling adjustment based on the initial camera pose data and the adjacent camera pose data to obtain updated camera pose data, and performing bundling adjustment based on the preliminary sparse point cloud data and the adjacent sparse point cloud data to obtain updated sparse point cloud data;

[0048] According to the feature point matching data, the updated camera pose data and the updated sparse point cloud data, the camera pose estimation, the sparse point cloud generation, the bundle adjustment processing and the iterative processing are performed to obtain the sparse point cloud data and the camera pose data.

[0049] In the present invention, by gradually adding new images and point cloud data and performing global bundle adjustment, a high-precision three-dimensional sparse point cloud model can be constructed. Bundling adjustment and iterative processing ensure high-precision estimation of camera pose, reduce reprojection errors, and make three-dimensional reconstruction more accurate. Gradually adding image pairs and point cloud data reduces the computational complexity of initial data processing, making the entire process more efficient. Through comprehensive analysis and optimization processing of multiple images, the stability and robustness of the model are enhanced, and it can adapt to complex scene changes.

[0050] Optionally, S3 includes:

[0051] Depth map data is obtained by performing depth map calculation based on sparse point cloud data and camera pose data;

[0052] Depth map fusion is performed based on the depth map data to obtain dense point cloud data of the roadbed.

[0053] The conversion process from sparse point cloud to dense point cloud in the present invention greatly enhances the detail expression of the three-dimensional model, making the model closer to the real scene. Depth map fusion integrates the depth information of different perspectives, ensuring the complete coverage of the scene and avoiding the information loss problem caused by a single perspective. The optimization algorithms in the fusion process, such as noise suppression and outlier removal, improve the accuracy and quality of the dense point cloud. Dense point cloud data has a wide range of applications in fields such as terrain modeling, building structure analysis, deformation monitoring, etc., and is the core data for many computer vision and three-dimensional reconstruction tasks.

[0054] Optionally, the roadbed dense point cloud data includes first roadbed dense point cloud data and second roadbed dense point cloud data, and the first roadbed dense point cloud data and the second roadbed dense point cloud data are roadbed dense point cloud data of the same location at different times, and S4 includes:

[0055] Performing multi-scale neighborhood generation according to the first roadbed dense point cloud data and the second roadbed dense point cloud data to obtain first multi-scale neighborhood data and second multi-scale neighborhood data;

[0056] Performing distance calculation based on the first multi-scale neighborhood data and the second multi-scale neighborhood data to obtain variation data;

[0057] Extract change features according to the change amount data to obtain change feature data;

[0058] Deformation identification is performed based on the change characteristic data to obtain the roadbed deformation data for roadbed deformation monitoring operations.

[0059] The multi-scale neighborhood in the present invention can analyze the local features and changes of point cloud data at multiple scales, from macro to micro, and provide comprehensive change information. By considering neighborhoods of different scales, subtle changes and large-scale changes in the point cloud can be captured and described more accurately. Multi-scale analysis helps to reduce the impact of noise and improve the robustness of feature extraction and change identification. Distance calculation provides information on the amount of change of each point in the point cloud, and can accurately measure the spatial changes of the roadbed at different time points. The distance calculation includes the changes of all points and can capture the displacement or deformation at any position on the roadbed. The change feature data provides detailed information about the deformation of the roadbed and can reflect the changes in different areas. Using the change feature data, the deformation type and degree of the roadbed can be automatically identified, reducing the errors of manual intervention and subjective judgment. The deformation identification results can be used as the basis for roadbed maintenance and repair, which is helpful for decision-making and resource allocation.

[0060] Optionally, a roadbed deformation monitoring system is used to execute the roadbed deformation monitoring method as described above, and the roadbed deformation monitoring system includes:

[0061] The multi-view image feature point matching module is used to collect multi-view image data of the highway subgrade using a drone to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; and match feature points based on the multi-view image feature point data to obtain feature point matching data;

[0062] The motion recovery structure processing module is used to perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data;

[0063] Multi-view stereo matching module, used to perform multi-view stereo matching based on sparse point cloud data and camera pose data to obtain dense point cloud data of roadbed;

[0064] The multi-scale point cloud comparison module is used to perform multi-scale point cloud comparison based on the dense point cloud data of the roadbed to obtain the roadbed deformation data for roadbed deformation monitoring and deformation operations.

[0065] The purpose of the present invention is that: in recent years, drone technology has developed rapidly. It has the characteristics of high efficiency, low cost, low risk, and strong accessibility to data acquisition capabilities, and is currently widely used in the field of highway surveying. The present invention uses drone technology to collect multi-view images of highway roadbeds, which greatly improves the efficiency of data acquisition and reduces the cost of roadbed monitoring. A high-precision three-dimensional point cloud model of the roadbed is established through the motion recovery structure algorithm (SfM) and the multi-view stereo matching algorithm (MVS). The three-dimensional point cloud model can more intuitively interpret the displacement and settlement changes of the roadbed from the three-dimensional space, and then grasp the changes in the stability state of the roadbed. Therefore, the advantage of the present invention is that it can intuitively reflect the size and position of the displacement and settlement changes of the roadbed, and the accuracy is high, so as to achieve an overall grasp of the deformation area of ​​the roadbed. This method can provide reliable roadbed monitoring data for the intelligent monitoring platform, and further assist in the construction of smart transportation on highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting implementations made with reference to the following drawings:

[0067] Figure 1 A flowchart showing a method for monitoring roadbed deformation according to an embodiment of the present invention is provided;

[0068] Figure 2 A flowchart showing a method for matching feature points of multi-view images according to an embodiment of the present invention is provided;

[0069] Figure 3 A flowchart showing a method for processing structure from motion according to an embodiment of the present invention is provided;

[0070] Figure 4 A flowchart of the steps of a multi-scale point cloud comparison method according to an embodiment is shown;

[0071] Figure 5 A flowchart showing a method for monitoring roadbed deformation based on oblique photography by unmanned aerial vehicles according to an embodiment of the present invention is shown;

[0072] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0073] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0074] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0075] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0076] Step 1: UAV multi-view image acquisition:

[0077] (1) Survey area

[0078] First, it is necessary to understand the traffic, topography and landforms of the survey area, check the roadbed environment on site, and determine the length and height of the roadbed, so as to facilitate the planning and design of subsequent routes. Secondly, confirm and mark the locations of objects that affect flight safety, such as signal towers and high-voltage lines, and confirm the take-off and landing sites in the operation area.

[0079] (2) Control point layout and measurement

[0080] As the basis for the registration of multi-phase point cloud data, ground control points must be laid out strictly according to the requirements. The reliability of control points directly affects the accuracy of roadbed monitoring results. The control points should be placed in clear and obvious places without deformation and not easily disturbed by environmental factors to ensure long-term preservation. In particular, when the exposed features of the roadbed are poorly discriminable and point selection is difficult, artificial signs with obvious reflective characteristics and easy corner extraction should be used as control points. The number of control points must be more than three to ensure the registration of multi-phase point cloud data. The measurement of ground control points usually uses a high-precision prism-free total station to measure their coordinate values.

[0081] (3) Route planning and design

[0082] When taking oblique photography with a drone, it is necessary to reasonably set the aerial photography parameters such as altitude, lateral overlap, heading overlap, and camera angle. Select the flight mode according to the environment around the roadbed. If necessary, the manual flight mode can be used instead of the automatic flight mode. In order to meet the monitoring accuracy requirements of the roadbed three-dimensional model, it is necessary to ensure that the image overlap is greater than 70%.

[0083] (4) Image data acquisition

[0084] The gimbal camera mounted on the drone can quickly and accurately obtain multi-view sequence image data of the roadbed in real time. The pilot implements oblique photogrammetry according to the established route design plan, always paying attention to the flight safety of the drone.

[0085] (5) Check image quality

[0086] Shooting quality check: Check the quality of photos taken by drones, including picture clarity, light, color reproduction and other aspects.

[0087] Data integrity check: Check whether the aerial photography data is complete, including whether the data files are complete and the data format is correct. Check by checking the number of data files, file size, metadata and other information.

[0088] Step 2: 3D point cloud model reconstruction:

[0089] (1) Image preprocessing;

[0090] (2) Extract feature points;

[0091] (3) Sparse point cloud reconstruction;

[0092] (4) Dense point cloud reconstruction.

[0093] Step 3: Roadbed deformation monitoring:

[0094] (1) Multi-phase point cloud data registration,

[0095] Point cloud precise registration based on ICP algorithm;

[0096] (2) Comparison of multi-phase point cloud models;

[0097] (3) Visual expression of displacement and settlement changes;

[0098] (4) Analysis of roadbed stability.

[0099] The M3C2 algorithm can detect changes in complex terrain directly on the point cloud without the need for meshing; and when calculating changes, it is less affected by spatial point density, surface roughness, and different sampling positions.

[0100] See also Figures 1 to 5 The present application provides a roadbed deformation monitoring method, the method comprising:

[0101] S1. Use a drone to collect multi-view image data of a highway subgrade to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; perform feature point matching based on the multi-view image feature point data to obtain feature point matching data;

[0102] Specifically, drones are used to collect images of the highway subgrade at different heights and angles to obtain multi-view image data. These image data should include different depression angles and viewing angles to ensure full coverage of the subgrade area.

[0103] Use computer vision technology (such as SIFT or SURF algorithm) to extract feature points from multi-view images. The specific process includes: image preprocessing (such as grayscale, denoising, etc.), detecting image feature points (such as corners or edges), and calculating the descriptors of feature points.

[0104] Feature point matching is performed based on feature point descriptors. The KD tree or FLANN algorithm is used to accelerate the feature point matching process, and the RANSAC algorithm is used to remove false matches. The output is feature point matching data, which contains a series of corresponding feature point pairs and their positions in the image.

[0105] S2. Perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data;

[0106] Specifically, the relative motion of the camera (such as relative position and posture) is calculated using feature point matching data, which is achieved by using an essential matrix or fundamental matrix estimation algorithm (such as the eight-point algorithm).

[0107] Using the prior camera parameters and feature point matching data, the absolute pose information (including position and rotation) of the camera at different shooting positions is calculated, and the camera pose data is output.

[0108] The bundle adjustment technology is used to fine-tune the camera pose and the position of feature points to obtain optimized sparse point cloud data and camera pose data.

[0109] S3, performing multi-view stereo matching based on the sparse point cloud data and the camera pose data to obtain dense point cloud data of the roadbed;

[0110] Specifically, based on the sparse point cloud data and camera pose data, the reference view and auxiliary view are selected, and a multi-view stereo matching algorithm (such as Semi-Global Matching, SGM) is used to generate a disparity map.

[0111] The disparity map is converted into a depth map and combined with the internal and external parameters of the camera to reconstruct dense 3D point cloud data.

[0112] To filter out noise and improve the accuracy of point clouds, possible methods include filtering and resampling point clouds to obtain high-quality dense point cloud data of the roadbed.

[0113] S4. Perform multi-scale point cloud comparison based on the dense point cloud data of the roadbed to obtain the roadbed deformation data for roadbed deformation monitoring and deformation operation.

[0114] Specifically, the dense point cloud data acquired at different times are aligned, and the ICP (Iterative Closest Point) algorithm is used to accurately align the point clouds.

[0115] By comparing the multi-phase point cloud data after registration, the displacement and deformation of the roadbed surface are calculated using multi-scale point cloud comparison technology (such as the M3C2 algorithm).

[0116] Analyze deformation data, detect obvious roadbed deformation areas, and generate deformation monitoring reports. The report should include detailed information such as the degree, location, and direction of deformation.

[0117] Specifically, the present invention is divided into three parts: highway roadbed multi-view image acquisition, three-dimensional point cloud model reconstruction and roadbed deformation monitoring.

[0118] Highway roadbed multi-view image acquisition includes: control point layout, control point measurement, survey area investigation, route planning and design, flight quality inspection, and flight data acquisition;

[0119] 3D point cloud model reconstruction includes: image preprocessing, feature point extraction, feature point matching, sparse point cloud reconstruction, and dense point cloud reconstruction;

[0120] Roadbed deformation monitoring includes: precise matching of point clouds, comparison of multi-phase point cloud models, displacement monitoring of cutting frame beams, and settlement monitoring of embankment and pavement.

[0121] In the present invention, an unmanned aerial vehicle is used to collect multi-view image data of a highway roadbed, and the aerial images are preprocessed, feature points are extracted, and feature points are matched. A motion recovery structure algorithm is used to obtain a sparse point cloud of the roadbed and a camera pose. Then, a multi-view stereo matching algorithm is used to obtain dense point cloud data of the roadbed. Subsequently, changes in multi-period point cloud data are compared based on a multi-scale model and a model point cloud comparison algorithm M3C2. Finally, deformation monitoring of the highway roadbed is achieved by visually expressing changes in displacement and settlement of the roadbed.

[0122] Optionally, the multi-view image data includes first multi-view image data and second multi-view image data, and the multi-view image feature point data includes first multi-view image feature point data and second multi-view image feature point data, and S1 includes:

[0123] S11, using a drone to collect multi-view image data of a highway roadbed through a preset first threshold height data to obtain first multi-view image data;

[0124] Specifically, the drone is used to collect multi-view image data at a preset first threshold altitude (a higher flight altitude).

[0125] The drone’s flight path and altitude are set, and image collection is performed through automatic flight or manual control to ensure comprehensive coverage from different angles. Image data includes high-angle bird’s-eye views, capturing a large-scale overview of the highway embankment.

[0126] S12, using the drone to collect multi-view image data of the highway roadbed through the preset second threshold height data to obtain second multi-view image data, wherein the first threshold height data is greater than the second threshold height data;

[0127] Specifically, the drone is used to collect multi-view image data at a preset second threshold altitude (lower flight altitude).

[0128] The drone’s flight path and altitude were set again to collect image data closer to the ground, which provided a higher-resolution and detailed view to capture the surface features of the highway bed.

[0129] S13, performing a first feature point detection on the first multi-view image data to obtain first multi-view image feature point data, and performing a second feature point detection on the second multi-view image data to obtain second multi-view image feature point data, wherein the first feature point detection and the second feature point detection are different feature point detection methods;

[0130] Specifically, a specific feature point detection algorithm, such as SIFT (Scale Invariant Feature Transform), is used on the first multi-view image data. The image data is converted into a grayscale image. Gaussian blur processing is performed to eliminate noise. Key points (such as edges and corners) in the image are detected. A feature descriptor is generated for each key point to form the first multi-view image feature point data.

[0131] Use another feature point detection algorithm, such as ORB (fast and rotation-invariant binary descriptor), on the second multi-view image data. Grayscale the image data. Use the ORB algorithm to detect feature points, focusing on rotation invariance and computational efficiency. Generate feature descriptors to form the second multi-view image feature point data.

[0132] S14, generating a descriptor according to the first multi-view image feature point data to obtain first descriptor data, and generating a descriptor according to the second multi-view image feature point data to obtain second descriptor data;

[0133] Specifically, the corresponding descriptor data is generated according to the feature point data of the first multi-view image. For each feature point, a feature vector is generated according to the gradient direction and amplitude of the surrounding pixel values. The feature vector is standardized to form the first descriptor data for subsequent matching.

[0134] According to the feature point data of the second multi-view image, the corresponding descriptor data is generated. For the ORB feature points, the binary descriptor based on rotation invariance is calculated. Each descriptor consists of a group of binary bits for efficient matching to form the second descriptor data.

[0135] S15. Perform matching calculation based on the first descriptor data and the second descriptor data to obtain feature point matching data.

[0136] Specifically, matching is performed based on the first descriptor data and the second descriptor data. KD tree or FLANN algorithm is used for preliminary matching to accelerate the search for matching pairs. RANSAC algorithm is used to further filter out false matches to ensure the reliability of matching pairs. Feature point matching data is output, including the location information and matching score of each pair of matching feature points.

[0137] In the present invention, the first threshold height provides a wider field of view, which is suitable for capturing global structures; the second threshold height provides a more detailed close-range perspective, which is suitable for capturing details and microscopic changes. Combining different viewing angle data at different heights can more accurately restore the true form of the three-dimensional scene, especially more effectively capturing details of complex terrain. Using different detection methods (such as SIFT, ORB, etc.), different types of feature points can be captured. One method may be good at detecting large-scale structural features, while another may be more suitable for extracting detail features. By combining multiple methods, the problem of missing feature points that may occur in a single method can be reduced, and the robustness of the detection results can be improved. Generating different descriptors based on different feature point data helps to better adapt to various image features and enhance the ability to distinguish feature points. Different descriptors can provide diverse feature representations for the same scene, which helps to improve the accuracy of matching. By matching and calculating descriptors from different heights and different methods, it is possible to make full use of multiple feature information and improve the accuracy and reliability of matching.

[0138] Optionally, the first feature point detection is specifically:

[0139] Performing color correction on the first multi-view image data to obtain color-corrected data;

[0140] Specifically, color correction is performed on the first multi-view image data to eliminate color deviations under different lighting conditions.

[0141] Adjust the white balance of an image to give it a more natural tone. Correct color using the grayscale world assumption or statistically based methods.

[0142] Adjust the brightness and contrast of the image to make it more suitable for subsequent processing. The specific method is to apply the gamma function to make the bright and dark parts of the image more balanced.

[0143] Perform histogram equalization on the RGB channels to enhance the overall color performance of the image and form color correction data.

[0144] Performing multi-scale spatial construction on the color correction data to obtain multi-scale spatial construction data;

[0145] Specifically, a series of blurred images are generated by applying Gaussian blur with different standard deviations to the image. Each blurred image represents a different scale space. The difference between blurred images of adjacent scales is calculated to form a Difference of Gaussians (DoG) pyramid. The local extreme points of each layer of the DoG pyramid correspond to feature point candidates at different scales, forming multi-scale space construction data.

[0146] Key point detection is performed on the multi-scale space construction data to obtain first multi-view image feature point data.

[0147] Specifically, in each layer of the DoG pyramid, the neighborhood of each pixel (3x3x3 cube, containing the neighborhood of the current layer, the previous layer, and the next layer) is checked to find the local extreme points. Each extreme point is accurately located, and the position, scale, and direction of the key points are refined using interpolation methods. Taylor series expansion is used to fit near the extreme points to determine the exact position and scale.

[0148] Each keypoint is assigned one or more main directions based on the gradient direction histogram of the keypoint neighborhood to ensure rotation invariance. The orientation of the keypoint is determined by the maximum value of the gradient direction.

[0149] The color correction in the present invention helps to eliminate color inconsistencies caused by different lighting conditions, camera parameters or atmospheric conditions, so that the colors of all images are more uniform and natural. By eliminating color differences, the stability of subsequent feature point detection can be enhanced, and feature point detection deviations caused by color differences can be avoided. Appropriate color correction can improve the contrast of the image, thereby more clearly highlighting the feature areas in the image. Multi-scale space construction can capture the features of the image at different scales, and can detect large-scale macro features as well as subtle local features. By analyzing images of different scales, the inaccuracy of feature point detection caused by scale changes can be reduced, and the robustness of feature points can be improved. Through key point detection, highly significant feature points can be extracted from multi-scale images, which is convenient for subsequent feature matching and three-dimensional reconstruction. Since key points are usually detected based on local information of the image (such as gradients, edges, etc.), the impact of noise on feature detection can be reduced.

[0150] Optionally, the multi-scale space construction data includes first multi-scale space construction data and second multi-scale space construction data, and the multi-scale space construction is specifically:

[0151] Performing texture multi-scale space construction on the color correction data to obtain first multi-scale space construction data;

[0152] Specifically, Gaussian blurring of different scales is applied to the color correction data to create a series of smooth images. Texture features such as gray-level co-occurrence matrix (GLCM), local binary pattern (LBP), etc. are extracted from the blurred images. A set of multi-scale texture images is constructed, each containing texture information of different scales, forming the first multi-scale space construction data.

[0153] Performing region division on the color correction data to obtain color correction region division data;

[0154] Specifically, an image segmentation algorithm (such as superpixel segmentation, K-means clustering, etc.) is used to divide the image into several regions. Each region represents a group of pixels with similar colors. A unique label is assigned to each segmented region to form color correction region division data.

[0155] Perform color feature extraction based on the color correction area division data to obtain regional color feature data;

[0156] Specifically, the color correction image is converted to a suitable color space (such as Lab, HSV, etc.) to more accurately analyze the color features. The average color, color distribution, color histogram and other features of each region are calculated. Statistical methods, such as calculating the mean and standard deviation, are used to describe the color characteristics of the region. The color feature data of each region are combined to form regional color feature data.

[0157] The color correction data is subjected to color multi-scale space construction according to the regional color feature data to obtain second multi-scale space construction data.

[0158] Specifically, multi-scale blur processing is performed on each region to simulate color changes at different scales. Similar to the difference Gaussian pyramid (DoG), a difference color pyramid is constructed to obtain multi-scale color features by calculating the color differences between different scales. A set of multi-scale color images is constructed, each of which contains color information at different scales, forming the second multi-scale space construction data.

[0159] The present invention can analyze the texture pattern in the image at multiple scales, from the global texture structure to the local detail features, and provide a comprehensive texture description. Multi-scale analysis can distinguish texture changes at different scales, especially in areas with complex textures and rich details, and provide more accurate feature extraction. By dividing the region, the features in the specific region can be analyzed intensively to avoid dilution of the overall image features. Regional division can help focus on local color changes when extracting color features, rather than the global color features of the entire image. Through regional color feature extraction, the color distribution and changes in different regions of the image can be captured, which helps to understand the content and scene of the image. By constructing a color multi-scale space, the changes in color features in the image at different scales can be analyzed, and the color features at large and small scales can be identified. Multi-scale color features can help maintain the consistency and accuracy of color features at different resolutions, and help to improve the accuracy of feature matching.

[0160] Optionally, the texture multi-scale space construction is specifically as follows:

[0161] Performing Gaussian blur on the color correction data to obtain Gaussian blurred data;

[0162] Specifically, different standard deviations (σ) are selected to construct Gaussian blur kernels. The Gaussian blur kernel is convolved with the image to obtain Gaussian blur data. This process makes the image have different blur levels at different scales.

[0163] Down-sample the Gaussian blurred data to obtain down-sampled data;

[0164] Specifically, the downsampling factor is selected by preset parameters, such as 2x2 window sampling, to reduce the image resolution to half of the original. The image size is reduced by skip sampling or average pooling to obtain downsampled data.

[0165] Extracting texture features from the downsampled data to obtain downsampled texture feature data;

[0166] Specifically, texture features are extracted using methods such as gray-level co-occurrence matrix (GLCM) or local binary pattern (LBP). Texture features such as contrast, homogeneity, entropy, etc. are calculated to obtain downsampled texture feature data.

[0167] Performing texture feature inversion on the downsampled data according to the downsampled texture feature data to obtain downsampled texture feature inversion data;

[0168] Specifically, a mathematical inversion operation, such as negation or complement, or a nonlinear transformation of the feature is used to perform an inversion calculation on each texture feature value to obtain downsampled texture feature inversion data.

[0169] Performing up-sampling processing on the down-sampled texture feature inversion data to obtain up-sampled data;

[0170] Specifically, bilinear interpolation or nearest neighbor interpolation is used for upsampling, and the inverted downsampled data is expanded to the original size to obtain upsampled data.

[0171] Subtraction processing is performed on the up-sampled data and the down-sampled data to obtain first multi-scale space construction data.

[0172] Specifically, pixel-by-pixel subtraction is performed to calculate the difference between the upsampled data and the downsampled data, and a difference map containing texture features of different scales is generated as the first multi-scale space construction data.

[0173] In the present invention, the color-corrected image data is subjected to Gaussian blur processing to reduce noise and details and obtain Gaussian blurred data. Downsampling reduces the resolution of the image, thereby reducing the amount of data and computational complexity, and is suitable for multi-scale analysis. By constructing a low-resolution version, the global features of the image and a larger range of patterns can be analyzed. Extracting texture features helps to identify details and local patterns in the image, especially on low-resolution images, where these features are more prominent. Through the inversion operation, the part opposite to the original feature can be enhanced or highlighted, providing a different perspective to observe the image content. Upsampling restores part of the original resolution so that the data can be compared with the high-resolution features. In the process of restoring the resolution, the information of the inverted features is combined to further highlight or smooth certain details. Subtraction processing can highlight the edges and details in the image, similar to the effect of the Laplace pyramid, and strengthen the change information in the image. By analyzing the difference between the upsampled data and the original data, the changes and feature distribution in the image can be better understood.

[0174] Optionally, the second feature point detection is specifically:

[0175] Performing image enhancement on the second multi-view image data to obtain image enhancement data;

[0176] Specifically, the brightness and contrast of the image are adjusted by the histogram equalization method to make the bright and dark areas of the image more balanced. The specific operation is to calculate the histogram of the image and then perform an equalization transformation on it. The Laplace operator is used to sharpen the image and enhance the edge features of the image. The specific operation is to convolve the Laplace operator with the image to enhance the details of the image. After applying the above image enhancement technology, image enhancement data with improved clarity and contrast is obtained.

[0177] Perform corner point detection on the image enhancement data to obtain corner point detection data;

[0178] Specifically, the Harris corner detection algorithm is used, and the specific steps are as follows: Calculate the gradient value of each pixel. Construct the Harris matrix and calculate the corner response function. Set the threshold, perform non-maximum suppression on the corner response function, and retain the local maximum point as the corner point. Corner detection data generation: Mark and save the detected corner point position to obtain the corner detection data.

[0179] The feature point directions are calculated for the angle detection data to obtain the second multi-view image feature point data.

[0180] Specifically, at each corner point of the corner detection data, the gradient direction and amplitude of its neighborhood are calculated. The specific operation is:

[0181] Calculate the gradient direction and amplitude of each pixel in the neighborhood of the corner point. Construct a direction histogram of the neighborhood of the corner point, with the angle as the horizontal axis and the gradient amplitude as the vertical axis, and count the sum of the gradient amplitudes in each direction. According to the direction histogram, find the direction corresponding to the maximum peak as the main direction of the corner point. If the secondary peak value is close to the main peak value, the secondary main direction can also be assigned. Combine the position of each corner point with the calculated direction information to obtain the feature point data of the second multi-view image.

[0182] Image enhancement in the present invention can improve the contrast and sharpness of the image, making the features easier to detect. By enhancing the local contrast, important image features such as corners and edges can be presented more clearly, which is convenient for subsequent feature point detection. Corner point detection provides rich local information and plays an important role in the global analysis and matching of image features. Calculating the direction of feature points helps to achieve the directional invariance of feature points, so that these feature points can still be matched when the image is rotated or flipped. By assigning a direction information to each feature point, the direction information can be used in the matching process to improve the accuracy and reliability of the matching. The direction information adds a dimension of description to each feature point, which helps to more accurately locate and identify feature points.

[0183] Optionally, S2 includes:

[0184] S21, selecting image pairs according to feature point matching data to obtain initial image pair data;

[0185] Specifically, the number of matches and the quality of matches in the feature point matching data are calculated, and an image pair with a large number of feature point matches and a more even distribution is selected. The best image pair is selected as the initial image pair to generate the initial image pair data.

[0186] S22, performing camera pose estimation according to the initial image pair data and preset drone camera internal parameter data to obtain initial camera pose data;

[0187] Specifically, the preset camera internal parameter data is used, including focal length, principal point coordinates, distortion coefficients, etc. The fundamental matrix or essential matrix is ​​calculated from the matched feature point pairs through the eight-point algorithm or other methods. The rotation and translation matrices of the camera are decomposed from the essential matrix to obtain the initial camera pose data.

[0188] S23, generating a sparse point cloud according to the initial image pair data and the initial camera pose data to obtain preliminary sparse point cloud data;

[0189] Specifically, the positions of 3D points are calculated from the matching points of two perspectives through a triangulation algorithm to generate preliminary sparse point cloud data.

[0190] S24, performing camera pose estimation and sparse point cloud generation according to the adjacent image pair data in the feature point matching data corresponding to the initial image pair data, to obtain adjacent camera pose data and adjacent sparse point cloud data;

[0191] Specifically, a process similar to the initial image pair is used to estimate the camera pose according to the data of the adjacent image pair to obtain the adjacent camera pose data. Sparse point cloud data is generated according to the adjacent image pair data and the adjacent camera pose data to obtain the adjacent sparse point cloud data.

[0192] S25, performing bundling adjustment according to the initial camera pose data and the adjacent camera pose data to obtain updated camera pose data, and performing bundling adjustment according to the preliminary sparse point cloud data and the adjacent sparse point cloud data to obtain updated sparse point cloud data;

[0193] Specifically, the objective function includes all camera positions, postures and positions of 3D points, minimizing the projection error, that is, the distance between the projection of the 3D point on the corresponding 2D image and the actual detection point. All camera postures and 3D point positions are jointly optimized using nonlinear optimization methods such as the Levenberg-Marquardt algorithm to obtain updated camera posture data and updated sparse point cloud data.

[0194] S26, performing camera pose estimation, sparse point cloud generation, bundle adjustment processing and iterative processing according to feature point matching data, updating camera pose data and updating sparse point cloud data to obtain sparse point cloud data and camera pose data.

[0195] Specifically, after the bundle adjustment, a new image pair is selected and the steps of camera pose estimation, sparse point cloud generation and bundle adjustment are repeated. The iterative process continues until all image pairs are processed or the projection error converges below the set threshold. The final sparse point cloud data and camera pose data are output.

[0196] In the present invention, by gradually adding new images and point cloud data and performing global bundle adjustment, a high-precision three-dimensional sparse point cloud model can be constructed. Bundling adjustment and iterative processing ensure high-precision estimation of camera pose, reduce reprojection errors, and make three-dimensional reconstruction more accurate. Gradually adding image pairs and point cloud data reduces the computational complexity of initial data processing, making the entire process more efficient. Through comprehensive analysis and optimization processing of multiple images, the stability and robustness of the model are enhanced, and it can adapt to complex scene changes.

[0197] Optionally, S3 includes:

[0198] Depth map data is obtained by performing depth map calculation based on sparse point cloud data and camera pose data;

[0199] Specifically, pixel matching is performed based on the camera pose and sparse point cloud data to find the corresponding point of each pixel in other images. Multi-view stereo matching algorithms such as Semi-Global Matching (SGM) or PatchMatch can be used.

[0200] Based on the matched pixels, calculate the disparity, which is the horizontal or vertical displacement of the pixels in different views. Convert the disparity value to the depth value , is the focal length, is the baseline distance, For disparity, calculate the depth of each pixel and generate depth map data.

[0201] Depth map fusion is performed based on the depth map data to obtain dense point cloud data of the roadbed.

[0202] Specifically, according to the camera pose data, each depth map is aligned to the same global coordinate system. The depth information in each depth map is converted to the same coordinate system using the camera's external parameters (pose). For multiple depth measurements at the same position, weighted average, maximum depth value, or minimum depth value are used for fusion. Weighted average can assign weights based on factors such as viewing angle and lighting conditions to reduce the impact of noise and errors. Redundant points or noise points generated during the fusion process are removed, and the point cloud is cleaned and optimized using methods including filtering, denoising, and resampling of the point cloud. Finally, dense point cloud data containing the three-dimensional coordinates and color information of each pixel is generated.

[0203] The conversion process from sparse point cloud to dense point cloud in the present invention greatly enhances the detail expression of the three-dimensional model, making the model closer to the real scene. Depth map fusion integrates the depth information of different perspectives, ensuring the complete coverage of the scene and avoiding the information loss problem caused by a single perspective. The optimization algorithms in the fusion process, such as noise suppression and outlier removal, improve the accuracy and quality of the dense point cloud. Dense point cloud data has a wide range of applications in fields such as terrain modeling, building structure analysis, deformation monitoring, etc., and is the core data for many computer vision and three-dimensional reconstruction tasks.

[0204] Optionally, the roadbed dense point cloud data includes first roadbed dense point cloud data and second roadbed dense point cloud data, and the first roadbed dense point cloud data and the second roadbed dense point cloud data are roadbed dense point cloud data of the same location at different times, and S4 includes:

[0205] S41, generating a multi-scale neighborhood according to the first roadbed dense point cloud data and the second roadbed dense point cloud data to obtain first multi-scale neighborhood data and second multi-scale neighborhood data;

[0206] Specifically, the neighborhood of a point is defined as a set of points within a certain range around it. The scale of the neighborhood can be determined by setting different radii or using the K-nearest neighbor (KNN) method. Select multiple different scale parameters, such as using different radii such as 1m, 5m and 10m, to generate corresponding multi-scale neighborhoods. According to each scale parameter, a multi-scale neighborhood point set is generated for each point in the first and second dense point cloud data, and the first multi-scale neighborhood data and the second multi-scale neighborhood data are obtained respectively.

[0207] S42, performing distance calculation according to the first multi-scale neighborhood data and the second multi-scale neighborhood data to obtain variation data;

[0208] Specifically, in the first and second multi-scale neighborhood data, corresponding point pairs are found for points at the same position, and these point pairs are aligned. For each pair of corresponding points, the Euclidean distance in the three-dimensional space is calculated. The distance difference between each pair of points is recorded to obtain the variation data.

[0209] S43, extracting change features according to the change amount data to obtain change feature data;

[0210] Specifically, analyze the change data and count the distribution of changes at different scales and regions. Use cluster analysis (such as K-means), threshold method or machine learning algorithm to extract features with obvious changes. This may include the magnitude of change, density of the change area and direction of change. Summarize the extracted change features to form change feature data.

[0211] S44, performing deformation identification according to the change characteristic data to obtain roadbed deformation data for performing roadbed deformation monitoring operations.

[0212] Specifically, deformation patterns such as settlement, uplift, and lateral displacement are identified based on the change characteristic data. Pattern recognition algorithms or rule-based methods can be used for identification. The identified deformations are classified and the type, degree, and location of the deformation are marked. Roadbed deformation data containing information such as deformation area, type, and degree are generated.

[0213] The multi-scale neighborhood in the present invention can analyze the local features and changes of point cloud data at multiple scales, from macro to micro, and provide comprehensive change information. By considering neighborhoods of different scales, subtle changes and large-scale changes in the point cloud can be captured and described more accurately. Multi-scale analysis helps to reduce the impact of noise and improve the robustness of feature extraction and change identification. Distance calculation provides information on the amount of change of each point in the point cloud, and can accurately measure the spatial changes of the roadbed at different time points. The distance calculation includes the changes of all points and can capture the displacement or deformation at any position on the roadbed. The change feature data provides detailed information about the deformation of the roadbed and can reflect the changes in different areas. Using the change feature data, the deformation type and degree of the roadbed can be automatically identified, reducing the errors of manual intervention and subjective judgment. The deformation identification results can be used as the basis for roadbed maintenance and repair, which is helpful for decision-making and resource allocation.

[0214] Optionally, a roadbed deformation monitoring system is used to execute the roadbed deformation monitoring method as described above, and the roadbed deformation monitoring system includes:

[0215] The multi-view image feature point matching module is used to collect multi-view image data of the highway subgrade using a drone to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; and match feature points based on the multi-view image feature point data to obtain feature point matching data;

[0216] The motion recovery structure processing module is used to perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data;

[0217] Multi-view stereo matching module, used to perform multi-view stereo matching based on sparse point cloud data and camera pose data to obtain dense point cloud data of roadbed;

[0218] The multi-scale point cloud comparison module is used to perform multi-scale point cloud comparison based on the dense point cloud data of the roadbed to obtain the roadbed deformation data for roadbed deformation monitoring and deformation operations.

[0219] The purpose of the present invention is that: in recent years, drone technology has developed rapidly. It has the characteristics of high efficiency, low cost, low risk, and strong accessibility to data acquisition capabilities, and is currently widely used in the field of highway surveying. The present invention uses drone technology to collect multi-view images of highway roadbeds, which greatly improves the efficiency of data acquisition and reduces the cost of roadbed monitoring. A high-precision three-dimensional point cloud model of the roadbed is established through the motion recovery structure algorithm (SfM) and the multi-view stereo matching algorithm (MVS). The three-dimensional point cloud model can more intuitively interpret the displacement and settlement changes of the roadbed from the three-dimensional space, and then grasp the changes in the stability state of the roadbed. Therefore, the advantage of the present invention is that it can intuitively reflect the size and position of the displacement and settlement changes of the roadbed, and the accuracy is high, so as to achieve an overall grasp of the deformation area of ​​the roadbed. This method can provide reliable roadbed monitoring data for the intelligent monitoring platform, and further assist in the construction of smart transportation on highways.

[0220] Therefore, from any point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the attached application documents rather than the above description, and it is intended that all changes falling within the meaning and scope of equivalent elements of the application documents are included in the present invention.

[0221] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A roadbed deformation monitoring method, characterized in that: The method comprises: S1. Use a drone to collect multi-view image data of a highway subgrade to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; perform feature point matching based on the multi-view image feature point data to obtain feature point matching data; S2. Perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data; S3, performing multi-view stereo matching based on the sparse point cloud data and the camera pose data to obtain dense point cloud data of the roadbed; S4. Perform multi-scale point cloud comparison based on dense point cloud data of the roadbed to obtain roadbed deformation data for roadbed deformation monitoring and deformation operation; The multi-view image data includes first multi-view image data and second multi-view image data, and the multi-view image feature point data includes first multi-view image feature point data and second multi-view image feature point data. S1 includes: Using the drone to collect multi-view image data of the highway roadbed through a preset first threshold height data to obtain first multi-view image data; Using the drone to collect multi-view image data of the highway roadbed through the preset second threshold height data to obtain second multi-view image data, wherein the first threshold height data is greater than the second threshold height data; Performing a first feature point detection on the first multi-view image data to obtain the first multi-view image feature point data, and performing a second feature point detection on the second multi-view image data to obtain the second multi-view image feature point data, wherein the first feature point detection and the second feature point detection are different feature point detection methods; Generate a descriptor based on the first multi-view image feature point data to obtain first descriptor data, and generate a descriptor based on the second multi-view image feature point data to obtain second descriptor data; Perform matching calculation based on the first descriptor data and the second descriptor data to obtain feature point matching data; The first feature point detection is specifically as follows: Performing color correction on the first multi-view image data to obtain color-corrected data; Performing multi-scale spatial construction on the color correction data to obtain multi-scale spatial construction data; Key point detection is performed on the multi-scale space construction data to obtain first multi-view image feature point data.

2. The method according to claim 1, characterized in that The multi-scale space construction data includes first multi-scale space construction data and second multi-scale space construction data. The multi-scale space construction is specifically as follows: Performing texture multi-scale space construction on the color correction data to obtain first multi-scale space construction data; Performing region division on the color correction data to obtain color correction region division data; Perform color feature extraction based on the color correction area division data to obtain regional color feature data; The color correction data is subjected to color multi-scale space construction according to the regional color feature data to obtain second multi-scale space construction data.

3. The method according to claim 2, characterized in that The construction of texture multi-scale space is as follows: Performing Gaussian blur on the color correction data to obtain Gaussian blurred data; Down-sample the Gaussian blurred data to obtain down-sampled data; Extracting texture features from the downsampled data to obtain downsampled texture feature data; Performing texture feature inversion on the downsampled data according to the downsampled texture feature data to obtain downsampled texture feature inversion data; Performing up-sampling processing on the down-sampled texture feature inversion data to obtain up-sampled data; Subtraction processing is performed on the up-sampled data and the down-sampled data to obtain first multi-scale space construction data.

4. The method according to claim 1, characterized in that: The second feature point detection is specifically as follows: Performing image enhancement on the second multi-view image data to obtain image enhancement data; Perform corner point detection on the image enhancement data to obtain corner point detection data; The feature point directions are calculated for the angle detection data to obtain the second multi-view image feature point data.

5. The method according to claim 1, characterized in that S2 include: Select image pairs based on feature point matching data to obtain initial image pair data; The camera pose is estimated based on the initial image pair data and the preset drone camera internal parameter data to obtain the initial camera pose data; Generate sparse point cloud according to initial image pair data and initial camera pose data to obtain preliminary sparse point cloud data; Perform camera pose estimation and sparse point cloud generation based on adjacent image pair data in the feature point matching data corresponding to the initial image pair data, and obtain adjacent camera pose data and adjacent sparse point cloud data; Performing bundling adjustment based on the initial camera pose data and the adjacent camera pose data to obtain updated camera pose data, and performing bundling adjustment based on the preliminary sparse point cloud data and the adjacent sparse point cloud data to obtain updated sparse point cloud data; According to the feature point matching data, the updated camera pose data and the updated sparse point cloud data, the camera pose estimation, the sparse point cloud generation, the bundle adjustment processing and the iterative processing are performed to obtain the sparse point cloud data and the camera pose data.

6. The method according to claim 1, characterized in that S3 include: Depth map data is obtained by performing depth map calculation based on sparse point cloud data and camera pose data; Depth map fusion is performed based on the depth map data to obtain dense point cloud data of the roadbed.

7. The method according to claim 1, characterized in that The roadbed dense point cloud data includes first roadbed dense point cloud data and second roadbed dense point cloud data, the first roadbed dense point cloud data and the second roadbed dense point cloud data are roadbed dense point cloud data at the same location at different times, and S4 includes: Performing multi-scale neighborhood generation according to the first roadbed dense point cloud data and the second roadbed dense point cloud data to obtain first multi-scale neighborhood data and second multi-scale neighborhood data; Performing distance calculation based on the first multi-scale neighborhood data and the second multi-scale neighborhood data to obtain variation data; Extract change features according to the change amount data to obtain change feature data; Deformation identification is performed based on the change characteristic data to obtain the roadbed deformation data for roadbed deformation monitoring operations.

8. A roadbed deformation monitoring system, characterized in that: Used to execute the roadbed deformation monitoring method according to claim 1, the roadbed deformation monitoring system comprises: The multi-view image feature point matching module is used to collect multi-view image data of the highway subgrade using a drone to obtain multi-view image data; extract feature points based on the multi-view image data to obtain multi-view image feature point data; and match feature points based on the multi-view image feature point data to obtain feature point matching data; The motion recovery structure processing module is used to perform motion recovery structure processing based on feature point matching data to obtain sparse point cloud data and camera pose data; Multi-view stereo matching module, used to perform multi-view stereo matching based on sparse point cloud data and camera pose data to obtain dense point cloud data of roadbed; The multi-scale point cloud comparison module is used to perform multi-scale point cloud comparison based on the dense point cloud data of the roadbed to obtain the roadbed deformation data for roadbed deformation monitoring and deformation operations.