A tilt monitoring device and method for coupling optical positioning and real-time updating

Through coupled optical positioning technology, the video stream is analyzed and combined with dense depth model, a three-dimensional deformation model of the foundation pit inclined tube was established, which solved the problems of low measurement accuracy and high cost in the existing technology, and achieved high-precision horizontal displacement of the foundation pit monitor.

CN114332362BActive Publication Date: 2024-11-26SOUTHEAST UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111545611.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-15
Publication Date
2024-11-26
Estimated Expiration
2041-12-15

AI Technical Summary

Technical Problem

The existing foundation pit horizontal displacement monitoring system has low measurement accuracy and cannot truly reflect the overall deformation of the inclined tube. It is also costly and has limitations in technical application.

Method used

Using a coupling optical positioning and real-time update of the inclination measurement monitoring device, the video stream is captured through a monocular lens, the relationship between frames is analyzed, the three-dimensional depth information of the key measurement points is obtained, and the dense depth model is combined to establish the inclination measurement tube. Three-dimensional deformation model realizes visual and digital accurate monitoring.

Benefits of technology

High-density measurement is achieved, with an error of up to 0.2 pixels (at least 0.05 mm level), which more realistically and comprehensively reflects the three-dimensional deformation of the inclined tube, reduces costs and improves monitoring accuracy and reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114332362B_ABST
    Figure CN114332362B_ABST
Patent Text Reader

Abstract

The present invention discloses an inclinometer monitoring device and method coupled with optical positioning and real-time updating, comprising: an inclinometer tube, a lifting end, a probe end and a control end. The inner wall of the inclinometer tube is provided with specially designed feature points to improve accuracy and facilitate decoding and positioning; the lifting end controls the lower speed of the probe through an automatic program; the probe end is equipped with a special lens and supporting facilities for obtaining image sequence information of the inclinometer tube and transmitting the data to the control end; the control end obtains a three-dimensional deformation model of the inclinometer tube and a traditional inclinometer monitoring curve based on an algorithm calculation of optical positioning and tracking technology. The present invention can more accurately monitor the deformation of the inclinometer tube, providing a new guarantee for the safety and stability of the project.
Need to check novelty before this filing date? Find Prior Art

Description

Technical field:

[0001] The invention relates to the technical field of geological disaster monitoring, and belongs to an inclinometer tube deformation monitoring device, in particular to an inclinometer monitoring device and method for coupling optical positioning and real-time updating. Background technology:

[0002] With the acceleration of urbanization, the development of urban underground space has entered a rapid development stage. As an important component of the development process, foundation pit engineering can cause serious hazards such as damage to buildings, instability, and even casualties within the affected area if not handled with care. Therefore, the construction principle of "safety first" must be implemented. In foundation pit engineering, the horizontal displacement of the foundation pit is the most important factor affecting the safety of the foundation pit, so it is very important to do a good job in monitoring the horizontal displacement of the foundation pit.

[0003] Monitoring the deformation of the inclinometer tube can effectively reflect the horizontal displacement of the foundation pit. The monitoring system is generally composed of the inclinometer tube and the inclinometer. According to the specification requirements, the portable inclinometer measures every 500mm by manually pulling the inclinometer probe. The measuring points are discontinuous and the measurement accuracy is low. It cannot truly reflect the overall deformation of the inclinometer tube. In addition, the instrument and labor costs are high, and there are certain technical application limitations.

[0004] Visual SLAM technology under the optical positioning and tracking technology system is an emerging map construction technology. It obtains surrounding environmental data through the camera, and estimates its own motion trajectory and constructs the environment based on the obtained data. It is divided into feature point method and direct method according to whether it is necessary to extract image features. The feature point method first obtains the relationship between key frames through feature matching, and then calculates the essential matrix (or homography matrix) between key frames based on the matching results using methods such as epipolar geometry, and performs matrix decomposition to obtain the motion trajectory of the camera. Finally, based on the motion relationship of the key frames, a three-dimensional model of the environment is assembled through error optimization and other methods.

[0005] Therefore, how to use visual SLAM technology to achieve high-density measurement and combine key measuring points to monitor the deformation of the inclinometer casing is a direction worthy of attention. Summary of the invention:

[0006] The present invention aims to provide an inclinometer monitoring device and method that couples optical positioning and real-time updating. Based on optical positioning and tracking technology, it tracks the video stream (observation image sequence) shot by a monocular lens, analyzes the relationship between frames, obtains the three-dimensional depth information of key measuring points, and establishes an inclinometer curve; and combined with a dense depth model, it obtains a three-dimensional deformation model of the inclinometer tube, realizing visual and digital precise monitoring.

[0007] A coupled optical positioning and real-time updating inclinometer monitoring device comprises an inclinometer tube, a lifting end, a probe end and a control end. The inclinometer tube is inserted underground, the lifting end controls the probe end to be dropped from top to bottom in the inclinometer tube, the control end collects information in the inclinometer tube through the probe end, the inner wall of the inclinometer tube is provided with specially designed feature points, including dot-shaped coded marks and ordinary marks, the probe end collects an image sequence containing the dot-shaped coded marks and the ordinary marks, and the control end analyzes a three-dimensional deformation model of the inclinometer tube.

[0008] As a further preferred scheme, the point-coded mark uses the identification information of the direction positioning point, the height positioning point and multiple positioning marks to preset information. The multiple positioning marks are distributed around the direction positioning point and the height positioning point, and the cross-ratio invariance distribution principle of the projection theorem is adopted; the ordinary mark uses the identification information with characteristic distinction arranged in random positions to preset information.

[0009] As a further preferred solution, a group of dot-shaped coded marks is arranged every 500 mm on the inner wall of the inclinometer tube, and ordinary marks are randomly distributed on the rest of the inner wall.

[0010] As a further preferred scheme, a point coding mark group includes four point coding marks arranged circumferentially at the same height in the inclinometer tube, the height positioning points of a point coding mark group are marked with the same height number, and the direction positioning points of a point coding mark group are marked with position numbers in four directions in sequence, and the order and direction reference of the position numbers of all point coding mark groups in the inclinometer tube are the same.

[0011] As a further preferred solution, the ordinary sign uses randomly arranged circles and identification information formed by randomly filling multiple areas within the circles to preset information. The circle is divided into an inner circle area and an outer ring area. The inner circle area and the outer ring area are equally divided into four areas, with a total of eight filled areas. The position of the ordinary sign is randomly arranged, and one of the eight filled areas within the ordinary sign is randomly filled as a sign.

[0012] As a further preferred solution, the probe end includes a protective shell, the lower part of the protective shell is a bent part, the protective shell has a built-in image guide bundle, and the bottom end of the protective shell is provided with a camera connected to the image guide bundle, the camera is used to collect and transmit point coded marks and ordinary mark information in the inclinometer tube, and the protective shell is also provided with a data transmission center connected to the camera, the data transmission center is used to temporarily store image sequence information, and the camera is also equipped with a lighting device.

[0013] A monitoring method for an inclinometer monitoring device coupled with optical positioning and real-time updating, wherein the camera adopts a fisheye lens, the camera imaging model is a pinhole imaging model, the camera distortion model adopts a FOV model, and its internal parameters include the focal length f of the camera x-axis x , the focal length of the camera on the y axis is fy , the x-coordinate c of the image principal point in the pixel coordinate system x , the y coordinate c of the image principal point in the pixel coordinate system y , and the field of view angle ω;

[0014] When the lens captures an image sequence, the ideal conversion relationship between the real coordinates of each pixel and the pixel coordinates is shown in Formula 1:

[0015] ZP p =K(R+t)P w (1)

[0016] Where Z is the scale factor; P p is the pixel coordinate system, recorded as (u,v,1) T ; K is the internal parameter matrix; R is the rotation matrix; t is the translation vector, R and t are collectively called the external parameter matrix; P w is the world coordinate system, denoted as (U, V, W, 1) T ; This formula shows that the world coordinate system of any point can be obtained by the external parameter matrix, that is, rotation and translation, the internal parameter matrix conversion and scale scaling, and the pixel coordinates of the point on the photo. The core idea of ​​the algorithm of the present invention is to inversely solve the coordinate transformation relationship;

[0017] The monitoring method comprises the following steps:

[0018] Step 1: Arrange a group of dot-shaped coded markers and common markers in the inclinometer tube, and update the internal parameters of the lens through the camera lens correction algorithm to eliminate the influence of lens distortion on accuracy;

[0019] The arrangement method of the dot-coded marking is as follows:

[0020] Multiple positioning marks are arranged at corresponding positions in the point-shaped coding mark. Their function is to identify the point-shaped coding mark based on the cross-ratio invariance of the positioning mark through the characteristic inclinometer curve calculation algorithm after the image is input; the height positioning point stores the distance of the point-shaped code from the pipe mouth, and each 500mm is a node. The red coding point information corresponds to the height positioning point, and the yellow coding point information corresponds to the direction positioning point. The black solid dot is the positioning mark, for example: 500mm away from the pipe mouth, the No. 1 position of the red coding point is selected and painted red, 1000mm away from the pipe mouth, the No. 2 position of the red coding point is selected and painted red, and so on; the direction positioning point stores the circumferential information of the point-shaped code, for example, if the point-shaped code is located on the A surface of the inclinometer tube (an artificially defined direction), the yellow code will be painted yellow, and ABCD represents four directions; then the characteristic inclinometer curve calculation algorithm first verifies the cross-ratio invariance of the positioning point to locate a certain point-shaped code, and then identifies the corresponding red coding point information and yellow coding point information through the target recognition module of the characteristic inclinometer curve calculation algorithm;

[0021] The lens correction algorithm is based on Zhang Zhengyou's camera calibration method (chessboard black and white grid method), and its mathematical principle is as follows:

[0022] Solve the internal parameter matrix K:

[0023] Paste the black and white chessboard on a flat white wall, set W = 0, and according to the matrix conversion of formula (1), the product of the internal parameter matrix and the external parameter matrix must be a 3×3 matrix H, so formula (1) can be simplified to formula (2);

[0024]

[0025] According to the homogeneity of the H matrix, it is simplified into equation (3) and equation (4);

[0026]

[0027]

[0028] From the above formula, we can know that when the actual coordinates and pixel coordinates of the black and white chessboard are greater than or equal to 4 pairs, the H matrix can be obtained;

[0029] Select the first column R1 and the second column R2 of the rotation matrix R. Due to the unit orthogonality of the rotation matrix, we can get:

[0030]

[0031]

[0032] Substituting the relationship between H matrix and R1, R2 into equations (3) and (4), we can obtain:

[0033]

[0034]

[0035] Combine the obtained H matrix and the above constraints to solve K first -T K -1 Matrix, and then decompose the internal parameter matrix K according to the internal parameter matrix as an upper triangular matrix;

[0036] Distortion elimination parameters:

[0037] When the actual image is converted, it will be distorted to a certain extent due to issues such as lens model. This algorithm considers both radial distortion and axial distortion, and its mathematical expression is formula (7). By substituting the coordinates of each corner point after distortion and the coordinates before distortion into formula (1), the least squares method is used to calculate the distortion parameters.

[0038]

[0039] Since the parameters solved in the two parts affect each other, all parameters are iteratively optimized based on the LM algorithm;

[0040] Step 2: Lower the probe end and fix the lifting end to prevent the instrument from becoming unstable during the automatic inclinometer measurement.

[0041] Step 3: Start the probe end and the lifting end. The lifting end controls the probe end to be lowered. The probe end collects image sequences through the lens, transmits the data to the control end, and backs it up in the data transmission center (5);

[0042] Step 4: The front end extracts key frames based on the SLAM matching algorithm, performs feature extraction and matching to calculate the pose (R and t) and transmits it to the back end;

[0043] The main framework of the SLAM-based matching algorithm is the ORB-SLAM2 algorithm, which is divided into feature matching and depth calculation;

[0044] Feature Matching:

[0045] After the image sequence is transmitted from the probe end, the algorithm compares the overlap rate between frames and adaptively selects key frames; then the ORB feature points of the key frames are extracted through features, and the epipolar constraint is used for initial pairing. After the initialization is completed, the ICP and P3P algorithms are used for matching; the epipolar constraint principle is to consider a certain three-dimensional point P in space and the frames at two moments; the world coordinates of P are (X, Y, Z), and its coordinates in the camera coordinate system of the two frames are P c1 and P c2 , the camera's intrinsic parameter matrix is ​​K, the rotation matrix R and translation vector t of the current frame are known, and we can get:

[0046]

[0047] This algorithm sets the threshold of the epipolar constraint to 0.001. If two pixels meet the error of the epipolar constraint, they are considered to be paired successfully;

[0048] After the features of the two images are matched, the EPNP of Open CV is used to calculate the position and posture of the lens. The mathematical principle is to find the best camera position by repeatedly iterating formula (8) so that the reprojection error between key frames is minimized.

[0049]

[0050] Among them, T is the Lie group representation of the camera pose, which represents R, t; P i is the spatial point coordinate; u i The pixel coordinates for its projection;

[0051] Depth calculation:

[0052] Depth calculation is to restore the depth information of the matched feature points. Its mathematical principle is triangulation. It considers a three-dimensional point P in space and two frames at two moments. The world coordinates of P are (X, Y, Z). Its homogeneous coordinates in the pixel coordinate system of the two frames are P and p1 (x1, y1, 1) and P p2 (x2, y2, 1), the camera's intrinsic parameter matrix is ​​K, the reference frame's rotation matrix R and translation vector t are known, and we can get:

[0053]

[0054] In formula (9), other quantities can be obtained through previous calculations, so the only unknown quantity left is Z, so the true three-dimensional information of the spatial point can be restored;

[0055] Step 5: The backend builds a sparse anchor point model based on the SLAM matching algorithm. The sparse anchor point model is established through the spatial relationship between the pose and the feature points and the motion relationship between the frames. The three-dimensional coordinates (position and depth) of the feature points are unified to the geodetic coordinates through the spatial relationship between the pose and the feature points, and then the model is spliced ​​through the motion relationship between the frames.

[0056] After obtaining the real coordinates of all feature points of the key frame and the position and posture of the probe (lens), all coordinates can be unified in the geodetic coordinate system through a simple coordinate transformation to complete the establishment of the sparse anchor point model;

[0057] Step 6: The front end includes the functions of traditional inclinometers based on the characteristic inclinometer curve calculation algorithm; it identifies key measuring points and further processes the information of key measuring points to calculate the deformation ratio of each pair of key measuring points, and the drawing software updates the curve graph;

[0058] The characteristic inclinometer curve calculation algorithm is to identify the point code by verifying the cross ratio invariance of the perspective projection of the point code mark for all the identified feature points, and then extract the mark points; since the positioning points and code points occupy fewer pixels, the centroid method is generally used to obtain the meaning of the mark points, and the threshold is 0.05. The mathematical formula is as follows:

[0059]

[0060] Among them, (i k ,j k ) represents the pixel coordinates, indicates (i k , j k ) at the pixel value;

[0061] Step 7: The back end calculates the 3D deformation model of the inclinometer casing based on the 3D model reconstruction algorithm;

[0062] Based on the 3D model reconstruction algorithm, the same key frames are selected by comparing the overlap rate between frames, and each pixel of a subset of all key frames is triangulated (see step 4 for explanation), and the 3D real coordinates of each pixel are calculated to obtain a dense model; then, through the pose information transmitted by the front end, and the feature points in the sparse anchor model updated by the front end are used as anchor points, the dense model is mapped and aligned with the sparse anchor model, and the error is evaluated by RMSE;

[0063] Step 8: Repeat steps 4 to 7 until the probe end is lowered to the bottom of the hole. It should be noted that step 7 starts running in parallel with step 3 after the image sequence is acquired to a certain extent, and then waits for step 5 to be transferred to the sparse anchor point model.

[0064] Step 9: The lifting end controls the probe end to pull up, and the probe end collects a return image sequence. The control end corrects the accuracy of the three-dimensional deformation model based on the accuracy correction algorithm. It should be noted that the accuracy correction algorithm is kept running throughout the entire control end calculation process, and this step is only a special accuracy correction step.

[0065] The precision correction algorithm includes: loop detection and the error function used in the calculation of each step;

[0066] Loop detection:

[0067] The bag-of-words model is used, that is, the feature points of the key frame are divided into words, and then the words appearing in the current frame and all previous frames are represented by description vectors (+1 if the word appears, and 0 if it does not appear). By comparing the description vectors of two frames, the similarity between the two frames can be defined. The mathematical formula is shown in formula (11). After many experiments, it is proved that loop detection can improve the accuracy of the model.

[0068]

[0069] Among them, a and b are description vectors; w is the sum of all words in the bag of words;

[0070] Step 10: The lifting end is lifted to the hole mouth to complete a positive measurement, and then the head is turned by the rotating mechanism, and steps 3 to 9 are repeated to achieve a reverse measurement;

[0071] Step 11: After the forward and reverse measurements are completed, turn off the instrument and save the inclinometer model.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] 1) The present invention uses SLAM technology to reconstruct the three-dimensional deformation model of the inclinometer tube to calculate the deformation degree of the inclinometer tube wall, and the error is at most 0.2 pixel error (at least 0.05 mm level). Considering the error accumulation caused by insufficient data of traditional inclinometer, this improvement can more truly and comprehensively reflect the three-dimensional deformation of the inclinometer tube.

[0074] 2) The present invention adds a small lighting system such as a shadowless lamp, which can avoid the influence of image features such as shadows, and can improve the overall model accuracy to form a trust model.

[0075] 3) According to the deformation characteristics of the inclinometer tube (plastic deformation, tube wall depression, etc.), the probe part is designed to be conical, and a certain soft support is used to keep the probe centered to obtain a wider field of view, while also effectively avoiding the problem of probe hole sticking.

[0076] Its shortcomings and improvement measures: Although the present invention needs to process more data, the parallel processing based on GPU can greatly reduce the processing time; the cost of the present invention is currently slightly lower than that of traditional inclinometers, and with the development of hardware, the supporting costs will drop significantly.

[0077] In general, the data measured by the inclinometer tube deformation monitoring method provided by the embodiment of the present invention is accurate and reliable, can more realistically reflect the actual deformation of the inclinometer tube, is convenient for monitoring the horizontal displacement of the foundation pit, and can more simply and accurately control the safety level of the foundation pit project. Description of the drawings:

[0078] Figure 1 It is a schematic diagram of the device of the present invention;

[0079] Figure 2 It is a schematic diagram of the structure of the probe end of the present invention;

[0080] Figure 3 A schematic diagram of the characteristic design of the present invention;

[0081] Figure 4 It is a system framework diagram of the present invention;

[0082] Figure 5 It is a partial algorithm flow chart of the present invention. Specific implementation method:

[0083] To make the purpose and technical advantages of the present invention clearer, the present invention will be further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the scope of the present invention.

[0084] If no specific conditions are specified in the examples, the experiment was carried out under normal conditions; if no manufacturer is specified for the instruments used, they can be conventional products purchased from the market; all technical terms used have the same meanings as those commonly understood by those skilled in the art of the present invention. The above is only for better description of the specific embodiments and is not intended to limit the present invention.

[0085] A device and method for monitoring inclination coupled with optical positioning and real-time updating, as shown in its schematic diagram Figure 1 As shown;

[0086] The whole device is mainly divided into: inclinometer tube 6, lifting end 7, probe end 8 and control end 9. Specially designed feature points are arranged on the inner wall of the inclinometer tube; the lifting end uses automatic lifting technology to precisely control indicators such as the probe descent speed; the probe end includes an in-tube lighting system, a monocular lens and its supporting facilities to ensure the accuracy of the video; the control end uses the following other algorithms to establish a three-dimensional deformation model of the inclinometer tube and extract characteristic inclinometer curves to achieve visual and accurate monitoring.

[0087] The present invention achieves the above-mentioned purpose through the following technical solutions: the present invention includes a lens correction algorithm, a feature point design scheme, a matching algorithm based on SLAM, a feature inclinometric curve calculation, a three-dimensional deformation model reconstruction algorithm, and an accuracy correction algorithm.

[0088] 1) Lens correction algorithm: used to calibrate the lens internal parameters and eliminate the impact of lens distortion on accuracy.

[0089] 2) Feature point design scheme: Use dot-coded marks and ordinary marks to design key measurement points and ordinary feature points respectively to improve matching speed and reduce false matching rate.

[0090] 3) SLAM-based matching algorithm: mainly includes feature matching, depth calculation and sparse map construction. Feature matching: select key frames → extract features → sparse mapping matching → optimize and eliminate false matches → probe pose and image feature points; depth calculation: calculate the depth information of feature points through triangulation and multi-frame optimization; sparse map construction: integrate all feature points in the same coordinate system based on pose data to establish a sparse anchor point model.

[0091] 4) Characteristic inclinometer curve calculation: It is divided into key measurement point extraction and curve calculation. Key measurement point extraction: Identify the coded information of all characteristic points, select key measurement points through positioning identification, and save their coded semantic information and depth information to TXT files. Curve calculation: Further process the information of key measurement points, and generate characteristic inclinometer curves (including traditional inclinometer monitoring curves) based on the pipe mouth point.

[0092] 5) 3D model reconstruction algorithm: including model construction and model alignment. Model construction: set frame overlap rate → select reconstruction key frames and their subsets → calculate the depth of each pixel in the selected subset → multi-objective optimization → build a dense depth model. Model alignment: based on the 3D information of key measuring points, align the dense depth model with the sparse model to form a 3D deformation model of the inclinometer casing.

[0093] 6) Precision correction algorithm: In order to ensure the accuracy of information calculation, BA optimization and loop detection are used during the calculation to ensure that the deformation accuracy can reach the sub-mm level.

[0094] In addition, the present invention also provides a method for estimating the three-dimensional spatial shape of the inclinometer casing, that is, by exporting the probe posture data, the three-dimensional spatial shape of the side oblique penetration can be roughly obtained.

[0095] The dot-shaped coding marks (key measuring points) are arranged every 500mm in the inclinometer tube. Their functions include the functions of the traditional inclinometer curve and can reflect the general deformation of the inclinometer tube. The ordinary marks (ordinary feature points) are arranged every 100mm on the other pipe walls. Their functions are to increase the number of feature points on the pipe wall for matching.

[0096] The point-shaped coding mark (key measuring point) consists of 5 positioning points and 14 coding points that distinguish different coding marks. It is rotationally invariant and can be arranged at any angle, such as Figure 3 a (point coding mark). The positioning point locates each point coding mark block based on the cross ratio invariant property of perspective projection, and the coding points are distributed in different design positions to store the information of key measuring points. The height positioning point indicates the vertical distance of the key measuring point from the pipe mouth, and the direction positioning point indicates the circumferential position of the key measuring point. By changing the number, position and combination relationship of the coding points, each marking point is given a unique digital identifier according to different design coordinates, forming a coding system containing semantic information.

[0097] The common mark is composed of a 1 / 8 solid circle, which is convenient for feature point extraction and processing. In the layout process, it is laid out between adjacent coded marks in an irregular manner (random layout, arbitrary rotation, etc.), such as Figure 3 b (normal sign) as shown.

[0098] The lifting end is composed of a fixed-speed lifting device and its supporting facilities. The speed is matched with the model accuracy and is divided into Type I, Type II and manual. Type I: Parallel modeling, fast speed and short time consumption, suitable for routine monitoring. Type II: Parallel modeling returns for loop detection to ensure the accuracy of modeling, suitable for precise monitoring. Manual: When an accident occurs, the lower probe can be manually lifted in time to prevent the probe from being damaged.

[0099] The probe end Figure 2As shown in the figure, it consists of a small lighting system, a monocular lens and its supporting facilities. The small lighting system uses a shadowless light ring to reduce the impact of shadows on feature matching. The monocular lens uses a computer vision special lens similar to the UI series of German IDS company to ensure recognition accuracy. The supporting equipment includes a data storage center for temporary storage of video streams (observation image sequences) to prevent data loss due to long line length or equipment aging.

[0100] The control end is the most innovative part of the present invention. According to the engineering characteristics of the inclinometer tube, the SLAM technology is improved and upgraded with an algorithm group to ensure the formation of an accurate characteristic inclinometer curve and a three-dimensional deformation model.

[0101] System framework, such as Figure 4 As shown, Note: The single-line frame diagram is for collecting video streams; the dotted-line frame diagram is for the front-end part; the double-line frame diagram is for the back-end part, as follows:

[0102] 1) Track and acquire image sequences, and transmit the image sequences to the front end and back end respectively.

[0103] 2) The front end calculates the pose through key frame extraction, feature extraction and matching and transmits it to the back end. It uses ORB descriptors to extract and match feature points of the new frame and the previous frame, and uses a constant speed motion model to calculate the probe pose. The error evaluation uses the Huber loss function to minimize the reprojection error.

[0104] 3) The front end triangulates the feature points from the selected keyframes and calculates the depth information of the feature points. It uses a strong filter to remove outliers and further optimizes the depth information by BA to improve the accuracy of the calculation.

[0105] 4) The backend builds a sparse anchor point model based on the spatial relationship between pose and feature points and the motion relationship between frames. It unifies the three-dimensional coordinates (position and depth) of feature points into geodetic coordinates through the spatial relationship between pose and feature points, and then splices the model through the motion relationship between frames.

[0106] 5) The front end locates the key measuring points based on projection invariance and extracts the information of the key measuring points by identifying the category codes of the feature points. The information of the key measuring points is further processed to calculate the deformation ratio of each pair of key measuring points, which is saved in the algorithm tuple in chronological order, and the curve chart is updated through the drawing software.

[0107] 6) The backend reconstructs the 3D deformation model based on the pose and image. The dense model aims to calculate the depth of each pixel in the selected keyframe subset. To simplify the computational complexity, a subset of keyframes is selected for dense calculation. The detailed steps are as follows: A) Key frame selection: Given the overlap rate δ1 of the reconstructed key frame, when the overlap rate between the reconstructed model projection and the key frame i is less than δ1, the key frame i is selected as the reconstructed key frame; B) Subset selection: Based on the disparity threshold α1, sequential search is started from the reconstructed key frame i until the edge frame j with a disparity greater than the threshold α1 is found, and the edge frame j, the reconstructed key frame i and the intermediate frame are selected as the reconstructed subset; C) Subset optimization: Based on the de-redundancy threshold α2, the frames in the reconstructed subset are screened. If the disparity between the frame and its adjacent frames is lower than the de-redundancy threshold α2, the frame is deleted; D) 3D reconstruction: The inverse depth map is used to describe the depth information of the reconstructed subset, and the calculation accuracy is evaluated in combination with the ZNCC weight and the Huber regularization norm; E) Model alignment: The feature points in the sparse anchor model updated by the front end and the poses of the key frames are used as anchors to keep the dense model aligned with the sparse SLAM mapping.

[0108] Among them, the front-end and the back-end are modular names for the algorithm, both of which run in the control end. The front-end is mainly responsible for processing and associating the raw data transmitted from the probe end, such as extracting feature points of the image, matching features, associating frames with each other, and simple precision correction. The back-end is mainly responsible for solving the problem of solving and optimizing the model based on the data transmitted by the front-end; and in order to ensure the detection accuracy, this technology proposes a new precision algorithm in the pose calculation and 3D deformation model reconstruction stages.

[0109] When calculating the probe end posture, the inner and outer layer iteration method is used to estimate the basic matrix (used to decompose and calculate the posture). The algorithm flow is as follows: Figure 5 a, inner and outer layer iteration method, where the black background is the outer layer iteration and the white background is the inner layer iteration (feature matching point pairs are input and the inner point subset of the basic matrix is ​​output). The core of the algorithm is to randomly initialize the outer layer subset and use the 7-point method to perform rough calculations in the outer layer, traverse all sample points to expand the inner point subset, and pass the filtered inner point set and the basic matrix to the inner layer after judging the inner point confidence. The inner layer then passes the result to the outer layer through the 7-point or 8-point iterative calculation, and judges whether the accuracy is qualified through the inner point confidence test.

[0110] In the reconstruction stage of the 3D deformable model, in order to prevent the system error from increasing with the number of frames, the system controls the accuracy from three aspects. A) Raise the judgment threshold of the reprojection error. B) Use multi-sequence extended reconstruction. The main core idea is to divide the video stream (observation image sequence) into multiple single sequences for extended reconstruction, and control data exchange through a temporary data bus. Finally, align the multiple sequences and output a dense field model. The algorithm flow is as follows: Figure 5b (Multi-sequence extension reconstruction method). C) The method of combining depth of field and motion. The main core idea is to integrate known motion information and depth information according to the characteristics of the motion observation system, weaken the depth of field estimation error and the perspective change and motion estimation error caused by motion. The algorithm flow is as follows: Figure 5 c (Method of combining motion and depth of field).

[0111] Special note: During the model alignment process, the actual shape of the inclinometer casing and the undeformed model segment will be combined to restore the true depth information of the inclinometer casing; and combined with texture methods to form the final textured model.

[0112] In summary, the deformation model measured by the inclinometer tube deformation monitoring method and device combined with optical instant positioning and mapping update provided in the embodiments of the present invention is more accurate and reliable, and can achieve inclinometer tube deformation monitoring with an error of 0.2 pixels (at least 0.05 mm), ensuring the safety of foundation pit engineering.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An inclinometer monitoring device coupled with optical positioning and real-time updating, comprising an inclinometer tube, a lifting end, a probe end and a control end, wherein the inclinometer tube is inserted underground, the lifting end controls the probe end to be dropped from top to bottom in the inclinometer tube, and the control end collects information in the inclinometer tube through the probe end, characterized in that: The inner wall of the inclinometer tube is provided with specially designed characteristic points, including dot-shaped coding marks and common marks. The probe end collects an image sequence including the dot-shaped coding marks and the common marks, and the control end analyzes a three-dimensional deformation model of the inclinometer tube. The dot-shaped coded mark uses the identification information of the direction positioning point, the height positioning point and multiple positioning marks to preset information. The multiple positioning marks are distributed around the direction positioning point and the height positioning point, and the cross-ratio invariance distribution principle of the projection theorem is adopted; the ordinary mark uses the identification information with characteristic distinction arranged at random positions to preset information; A point-shaped coding mark group is arranged every 500 mm on the inner wall of the inclinometer tube, and ordinary marks are randomly distributed on the remaining inner walls; A point-coded mark group includes four point-coded marks arranged circumferentially at the same height in the inclinometer tube. The height positioning points of the four point-coded marks in a point-coded mark group are marked with the same height number. The direction positioning points of a point-coded mark group are marked with position numbers of four directions in sequence respectively. The order and direction reference of the position numbers of all point-coded mark groups in the inclinometer tube are the same. The ordinary sign uses randomly arranged circles and identification information formed by randomly filling multiple areas within the circles to preset information. The circle is divided into an inner circle area and an outer circle area. The inner circle area and the outer ring area are equally divided into four areas, with a total of eight filling areas. The position of the ordinary sign is randomly arranged, and one of the eight filling areas in the ordinary sign is randomly filled as a sign.

2. The inclination monitoring device coupled with optical positioning and real-time updating according to claim 1, characterized in that: The probe end comprises a protective shell (1), the lower part of the protective shell (1) is a bent part (2), the protective shell (1) has a built-in image guide bundle (3), the bottom end of the protective shell (1) is provided with a camera (4) connected to the image guide bundle (3), the camera (4) is used to collect and transmit point-shaped coded marks and common mark information in the inclinometer casing, the protective shell (1) is also provided with a data transmission center (5) connected to the camera (4), the data transmission center (5) is used to temporarily store image sequence information, and the camera (4) is also provided with a lighting device.

3. A monitoring method for a coupled optical positioning and real-time updated inclinometer monitoring device according to any one of claims 1-2, characterized in that: The camera (4) adopts a fisheye lens, the imaging model of the camera (4) is a pinhole imaging model, the distortion model of the camera adopts a FOV model, and its internal parameters include the focal length f of the camera x-axis. x , the focal length of the camera on the y axis is f y , the x-coordinate c of the image principal point in the pixel coordinate system x , the y coordinate c of the image principal point in the pixel coordinate system y , and the field of view angle ω; When the lens captures an image sequence, the ideal conversion relationship between the real coordinates of each pixel and the pixel coordinates is shown in Formula 1: ZP p =K(R+t)P w (1) Where Z is the scale factor; P p is the pixel coordinate system, recorded as (u,v,1) T ; K is the internal parameter matrix; R is the rotation matrix; t is the translation vector, R and t are collectively called the external parameter matrix; P w is the world coordinate system, denoted as (U, V, W, 1) T ; This formula shows that the world coordinate system of any point can be obtained by the external parameter matrix, i.e. rotation (R) and translation (t), internal parameter matrix conversion and scale scaling, and the pixel coordinates of the point on the photo can be obtained. The core idea of ​​the algorithm of the present invention is to inversely solve the coordinate transformation relationship; The monitoring method comprises the following steps: Step 1: Arrange a group of dot-shaped coded markers and common markers in the inclinometer tube, and update the internal parameters of the lens through the camera lens correction algorithm to eliminate the influence of lens distortion on accuracy; The arrangement method of the dot-coded marking is as follows: Multiple positioning marks are arranged at corresponding positions in the dot-shaped coding mark. Their function is to identify the dot-shaped coding mark based on the cross-ratio invariance of the positioning mark through the characteristic inclinometer curve calculation algorithm after the image is input; the direction positioning point stores the distance of the dot-shaped code from the pipe mouth, and every 500mm is a node; the direction positioning point stores the circumferential information of the dot-shaped code; then the characteristic inclinometer curve calculation algorithm first verifies the cross-ratio invariance of the positioning mark to locate a certain dot-shaped code, and then identifies the corresponding height positioning point information and direction positioning point information through the target recognition module of the characteristic inclinometer curve calculation algorithm; The lens correction algorithm is based on Zhang Zhengyou's camera calibration method, and its mathematical principle is as follows: Solve the internal parameter matrix K: Paste the black and white chessboard on a flat white wall, set W = 0, and according to the matrix conversion of formula (1), the product of the internal parameter matrix and the external parameter matrix must be a 3×3 matrix H, so formula (1) can be simplified to formula (2); According to the homogeneity of the H matrix, it is simplified into equation (3) and equation (4); From the above formula, we can know that when the actual coordinates and pixel coordinates of the black and white chessboard are greater than or equal to 4 pairs, the H matrix can be obtained; Select the first column R1 and the second column R2 of the rotation matrix R. Due to the unit orthogonality of the rotation matrix, we can get: Substituting the relationship between H matrix and R1, R2 into equations (3) and (4), we can obtain: Combine the obtained H matrix and the above constraints to solve K first -T K -1 Matrix, and then decompose the internal parameter matrix K according to the internal parameter matrix as an upper triangular matrix; Distortion elimination parameters: When the actual image is converted, it will be distorted to a certain extent due to issues such as lens model. This algorithm considers both radial distortion and axial distortion, and its mathematical expression is formula (7). By substituting the coordinates of each point after distortion and the coordinates before distortion into formula (1), the least squares method is used to calculate the distortion parameters. Since the parameters solved in the two parts affect each other, all parameters are iteratively optimized based on the LM algorithm; Step 2: Lower the probe end and fix the lifting end to prevent the instrument from becoming unstable during the automatic inclinometer measurement. Step 3: Start the probe end and the lifting end. The lifting end controls the probe end to be lowered. The probe end collects image sequences through the lens, transmits the data to the control end, and backs it up in the data transmission center (5); Step 4: The front-end extracts key frames based on the SLAM matching algorithm, performs feature extraction and matching to calculate the pose and transmit it to the back-end; The main framework of the SLAM-based matching algorithm is the ORB-SLAM2 algorithm, which is divided into feature matching and depth calculation; Feature Matching: After the image sequence is transmitted from the probe end, the algorithm compares the overlap rate between frames and adaptively selects key frames; then the ORB feature points of the key frames are extracted through features, and the epipolar constraint is used for initial pairing. After the initialization is completed, the ICP and P3P algorithms are used for matching; the epipolar constraint principle is to consider a certain three-dimensional point P in space and the frames at two moments; the world coordinates of P are (X, Y, Z), and its coordinates in the camera coordinate system of the two frames are P c1 and P c2 , the camera's intrinsic parameter matrix is ​​K, the rotation matrix R and translation vector t of the current frame are known, and we can get: This algorithm sets the threshold of the epipolar constraint to 0.

001. If two pixels meet the error of the epipolar constraint, they are considered to be paired successfully; After the features of the two images are matched, the EPNP of Open CV is used to calculate the position and posture of the lens. The mathematical principle is to find the best camera position by repeatedly iterating formula (8) so that the reprojection error between key frames is minimized. Among them, T is the Lie group representation of the camera pose, which represents R, t; P i is the spatial point coordinate; u i The pixel coordinates for its projection; Depth calculation: Depth calculation is to restore the depth information of the matched feature points. Its mathematical principle is triangulation. It considers a three-dimensional point P in space and two frames at two moments. The world coordinates of P are (X, Y, Z). Its homogeneous coordinates in the pixel coordinate system of the two frames are P and p1 (x1,y1,1) and P p2 (x2, y2, 1), the camera's intrinsic parameter matrix is ​​K, the reference frame's rotation matrix R and translation vector t are known, and we can get: In formula (9), other quantities can be obtained through previous calculations, so the only unknown quantity left is Z, so the true three-dimensional information of the spatial point can be restored; Step 5: The backend builds a sparse anchor point model based on the SLAM matching algorithm. The sparse anchor point model is established through the spatial relationship between the pose and the feature points and the motion relationship between the frames. The 3D coordinates of the feature points are unified to the geodetic coordinates through the spatial relationship between the pose and the feature points, and then the model is spliced ​​through the motion relationship between the frames. After obtaining the real coordinates of all feature points of the key frame and the position and posture of the probe, all coordinates can be unified in the geodetic coordinate system through a simple coordinate transformation to complete the establishment of the sparse anchor point model; Step 6: The front end includes the functions of traditional inclinometers based on the characteristic inclinometer curve calculation algorithm; it identifies key measuring points and further processes the information of key measuring points to calculate the deformation ratio of each pair of key measuring points, and the drawing software updates the curve graph; The characteristic inclinometer curve calculation algorithm is to identify the point code by verifying the cross ratio invariance of the perspective projection of the point code mark for all the identified feature points, and then extract the mark points; since the positioning points and code points occupy fewer pixels, the centroid method is generally used to obtain the meaning of the mark points, and the threshold is 0.

05. The mathematical formula is as follows: Among them, (i k ,j k ) represents the pixel coordinates, indicates (i k ,j k ) at the pixel value; Step 7: The back end calculates the 3D deformation model of the inclinometer casing based on the 3D model reconstruction algorithm; Based on the 3D model reconstruction algorithm, the same key frames are selected by comparing the overlap rate between frames, and each pixel of a subset of all key frames is triangulated, and the 3D real coordinates of each pixel are calculated to obtain a dense model; then, the dense model is mapped and aligned with the sparse anchor model through the pose information transmitted by the front end and the feature points in the sparse anchor model updated by the front end are used as anchor points, and the error is evaluated by RMSE; Step 8: Repeat steps 4 to 7 until the probe end is lowered to the bottom of the hole. It should be noted that step 7 starts running in parallel with step 3 after the image sequence is acquired to a certain extent, and then waits for step 5 to be transferred to the sparse anchor point model. Step 9: The lifting end controls the probe end to pull up, and the probe end collects a return image sequence. The control end corrects the accuracy of the three-dimensional deformation model based on the accuracy correction algorithm. It should be noted that the accuracy correction algorithm is kept running throughout the entire control end calculation process, and this step is only a special accuracy correction step. The precision correction algorithm includes: loop detection and the error function used in the calculation of each step; Loop detection: The bag-of-words model is used, that is, the feature points of the key frame are divided into words, and then the words appearing in the current frame and all previous frames are represented by description vectors. By comparing the description vectors of two frames, the similarity between the two frames can be defined. Its mathematical formula is shown in formula (11). After many experiments, it is proved that loop detection can improve the accuracy of the model. Among them, a and b are description vectors; w is the sum of all words in the bag of words; Step 10: Lift the lifting end to the hole mouth, complete a forward measurement, then turn 80 degrees, repeat steps 3 to 9, and achieve a reverse measurement; Step 11: After the forward and reverse measurements are completed, turn off the instrument and save the inclinometer model.