Roadway deformation real-time detection method and system based on binocular vision

Abstract

The invention discloses a roadway deformation real-time detection method and system based on binocular vision, and the method comprises the steps: firstly, installing and debugging a mining industrial binocular camera, and training an anchor rod end target detection algorithm; then real-time acquisition and preprocessing of images are carried out, and environmental noise is removed by using a self-adaptive median filter; and based on the preprocessed image, identifying and positioning the end of the anchor rod by using a trained target detection algorithm. A binocular vision technology is utilized to calculate three-dimensional coordinates of the end of the anchor rod, and feature matching and parallax calculation between images are carried out. And finally, comparing the three-dimensional coordinates of the ends of the anchor rods at different time points, and inputting the three-dimensional coordinates into a roadway deformation calculation module to obtain a deformation result of the roadway so as to realize real-time deformation monitoring and early warning of the roadway.

Description

Technical Field

[0001] The present invention relates to the technical field of tunnel deformation detection, and in particular to a binocular vision-based real-time tunnel deformation detection method and system. Background Art

[0002] As an important passage in a mine, the stability of the tunnel is directly related to the safety of mine production and the life safety of workers. The occurrence of tunnel deformation may lead to the loss of integrity of the surrounding rock structure, which in turn causes collapse accidents, posing a serious threat to personnel safety and equipment. In addition, tunnel deformation may also affect the normal operation of underground projects, causing transportation delays and increasing the cost of maintenance and reinforcement. Therefore, effective tunnel deformation detection can not only warn of potential dangers in advance, but also help take measures to prevent disasters and ensure the continuity and safety of mine production.

[0003] Traditional methods for measuring tunnel deformation mainly include laser rangefinders, roof delamination meters, and electronic total station measurements. These methods can meet engineering needs to a certain extent, but they still have many limitations. (1) Traditional detection methods require regular manual inspections and data recording, and cannot achieve continuous detection and real-time data updates. This limits the ability to respond quickly to sudden deformation events in the tunnel. (2) The data processing and analysis capabilities of existing detection methods often rely on manual labor, and cannot effectively analyze and interpret large amounts of detection data. This limits in-depth research and accurate prediction of tunnel deformation laws. (3) Traditional detection equipment has poor adaptability in harsh environments such as humidity and dust, and is prone to failure or performance degradation, affecting the reliability and stability of detection. (4) Many detection methods can only detect local areas of the tunnel and cannot achieve comprehensive coverage of the entire tunnel space. This limitation may lead to deformation in certain key areas being ignored. Summary of the invention

[0004] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a real-time detection method and system for tunnel deformation based on binocular vision, which can detect the spatial displacement of all anchor rod ends in the entire tunnel section in real time through target detection and binocular vision three-dimensional coordinate solution technology, and then calculate the convergence amount of the tunnel top and bottom plates and the convergence amount of the two sides, so as to infer the crushing and deformation of the tunnel surrounding rock.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] The present invention provides a real-time detection method for lane deformation based on binocular vision, comprising the following steps:

[0007] Step 1: Install and debug the binocular camera in the tunnel, and train the anchor end target detection algorithm;

[0008] Step 2: Use a binocular camera to collect images and perform image preprocessing;

[0009] Step 3: Use the pre-trained target detection algorithm to identify and locate the anchor end;

[0010] Step 4: Calculate the three-dimensional coordinates of the anchor bolt end using binocular vision three-dimensional imaging technology based on the two-dimensional positioning information;

[0011] Step 5: Calculate the spatial position changes of the anchor bolt ends at different times, and use this to solve the movement of the top and bottom plates and the two sides in different areas of the tunnel.

[0012] Preferably, in step 1, installing and debugging a binocular camera and training an anchor end target detection algorithm specifically includes the following steps:

[0013] Step 1.1: Install a binocular camera on the tunnel roof, and perform binocular camera calibration, distortion correction, and stereo correction debugging;

[0014] Step 1.2: Establish an anchor end image dataset and train an anchor end target detection algorithm.

[0015] Preferably, in step 2, a binocular camera is used to collect images and perform image preprocessing, which specifically includes the following steps:

[0016] Step 2.1, using a binocular camera to collect images;

[0017] Step 2.2, using an adaptive median filter to remove noise from the collected image;

[0018] Step 2.3: Manually mark the approximate area of ​​the anchor end as the region of interest.

[0019] Preferably, in step 4, the three-dimensional coordinates of the anchor end are calculated using binocular vision three-dimensional imaging technology according to the two-dimensional positioning information, which specifically includes the following steps:

[0020] Step 4.1, binocular camera image registration: including the steps of anchor rod end feature extraction in the target detection frame area, feature descriptor calculation, binocular camera image feature matching and output matching pair list;

[0021] Step 4.2, Calculate the three-dimensional coordinates of the anchor end: obtain the matching pairs of corresponding feature points according to the image registration, calculate the disparity through the matching pairs, and then calculate the depth information of the object using the disparity;

[0022] Step 4.3, coordinate system conversion: Convert the points in camera coordinates to a global or world coordinate system.

[0023] Preferably, in step 5, the spatial position changes of the anchor bolt ends at different times are calculated, and the top and bottom plate displacements and the two side displacements of different areas of the tunnel are solved based on the calculation, which specifically includes the following steps:

[0024] Step 5.1, calculation of spatial displacement of the anchor bolt end: use the Euclidean distance formula to calculate the spatial displacement of the anchor bolt end at different times;

[0025] Step 5.2, calculation of tunnel deformation: the overall stability and deformation of the tunnel are evaluated by monitoring the convergence of the top and bottom plates and the convergence of the two sides.

[0026] The present invention also provides a binocular vision-based real-time detection system for tunnel deformation, comprising:

[0027] An image acquisition unit, used for acquiring and processing image data from a binocular camera in real time;

[0028] The image preprocessing unit uses an adaptive median filter to remove environmental noise and interference to ensure image quality; by setting the region of interest, the calculation amount of the subsequent target detection algorithm is reduced;

[0029] Anchor end recognition unit, used to identify and locate the anchor end in the image, ensuring high accuracy and high reliability of target detection, and providing precise two-dimensional coordinate data of the anchor end for subsequent analysis;

[0030] The three-dimensional coordinate solving unit of the anchor end is used to calculate the three-dimensional coordinates of the anchor end in the world coordinate system, and accurately solve the position in the three-dimensional space through feature point matching and parallax calculation;

[0031] The tunnel deformation calculation unit is used to analyze the changes in the three-dimensional coordinates of the anchor bolt end and calculate the movement of the top and bottom plates and the two sides of the tunnel.

[0032] The beneficial effects of the present invention are:

[0033] Compared with the method of deformation detection of tunnels based on traditional sensors, the present invention has significant advantages. First, the system does not require preset monitoring points, and directly captures the entire tunnel through a binocular camera, providing higher accuracy and timeliness, while achieving a higher degree of automation. Secondly, compared with traditional deformation detection technology that relies on time-series distance measurement between point clouds, the present invention uses image recognition and stereo vision technology to perform fine characterization of the tunnel surface, which is more suitable for complex environments such as mine tunnels. Thirdly, the present invention uses deep learning algorithms, such as target detection models, to perform real-time automatic analysis of deformation data, which not only eliminates the subjective errors that may be caused by manual processing of measurement data, but also can accurately identify the specific location of the deformation in real time.

[0034] In addition, the equipment deployment of this system is simple and the maintenance cost is low. It is easy to deploy and use quickly in a mine environment, which greatly improves the efficiency and safety of underground inspections. In summary, the tunnel deformation detection technology based on binocular vision of the present invention realizes the ability to obtain the deformation position in the tunnel from end to end, greatly simplifies the workflow, and improves the detection efficiency and accuracy. Therefore, the present invention can not only realize real-time and accurate tunnel deformation monitoring, but also provide important technical support and decision-making basis for mine support and maintenance work. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 Schematic diagram of the process of the real-time detection method of tunnel deformation in an embodiment of the present invention;

[0037] Figure 2 It is a schematic diagram of the deployment of the region of interest of the anchor rod end in a specific embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of binocular image feature matching in a specific embodiment of the present invention;

[0039] Figure 4 It is a schematic diagram of solving the three-dimensional coordinates of the anchor rod end in a specific embodiment of the present invention;

[0040] Figure 5 It is a schematic diagram of the calculation of the amount of movement of the top and bottom plates of the tunnel and the amount of movement of the two sides according to a specific embodiment of the present invention.

[0041] Figure 6 Schematic diagram of the real-time detection system for tunnel deformation in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] like Figure 1-Figure 5As shown, this embodiment discloses a real-time detection method for tunnel deformation based on binocular vision, which can be applied to the real-time monitoring of the movement of the tunnel roof and floor and the movement of the two sides. First, install and debug the mining industrial binocular camera, and train the anchor end target detection algorithm. Then, real-time image acquisition and preprocessing are performed, and an adaptive median filter is used to remove environmental noise. Based on the preprocessed image, the trained target detection algorithm is used to identify and locate the anchor end. Using binocular vision technology, the three-dimensional coordinates of the anchor end are calculated, and feature matching and parallax calculations are performed between images. Finally, by comparing the three-dimensional coordinates of the anchor end at different time points and inputting them into the tunnel deformation calculation module, the deformation result of the tunnel is obtained, thereby realizing real-time deformation monitoring and early warning of the tunnel.

[0044] The specific steps include:

[0045] Step 1: Install and debug the binocular camera and train the anchor end target detection algorithm:

[0046] Step 1.1: Install a binocular camera on the tunnel roof, and perform debugging work such as binocular camera calibration, distortion correction, and stereo correction:

[0047] Choose industrial-grade cameras for mining. The binocular camera should be aimed at the tunnel roof area and installed on the top or side wall of the tunnel to ensure that it can cover the width and height of the entire tunnel. The debugging of the binocular camera mainly includes binocular camera calibration, distortion correction, and stereo correction steps.

[0048] Step 1.1.1, binocular camera calibration:

[0049] Select a calibration object with known geometric characteristics (usually a calibration plate or chessboard), and use a binocular camera to take multiple photos of the calibration object from different angles and distances. Use the stereo camera calibration toolbox in Matlab software for calibration, select 3 parameters for radial distortion, and use the default parameters for other parameters. After the calibration is completed, several key parameters such as internal parameters, external parameters, and distortion coefficients will be obtained. The internal parameters mainly include focal length f, optical center (c x ,c y ), the camera intrinsic parameter matrix K. The external parameters mainly include the rotation matrix R and the translation vector T. The distortion coefficients are mainly [k1, k2, k3, p1, p2], where k1, k2, k3 describe the radial distortion of the lens, and p1, p2 describe the tangential distortion of the lens.

[0050] Step 1.1.2, binocular camera distortion correction:

[0051] The goal of distortion correction is to correct the distorted image coordinates (x0, y0) to the undistorted image coordinates (x, y).

[0052] Radial distortion and tangential distortion can be expressed by the following formulas:

[0053]

[0054] Where (x′, y′) is the normalized image coordinate, (x0, y0) is the pixel coordinate before distortion correction; r 2 =(x ′ ) 2 +(y ′ ) 2 ; (x, y) is the pixel coordinate after distortion correction; k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients.

[0055] Step 1.1.3, binocular camera stereo calibration:

[0056] The main purpose of stereo rectification is to geometrically transform the images taken by two cameras (i.e., the left and right cameras) so that the corresponding point pairs are located in the same row in the two images, thereby simplifying the stereo matching process. The specific steps are as follows:

[0057] (1) Adjust the rotation matrices of the left and right cameras so that the image planes of the two cameras are on the same plane and the baseline is parallel to the image plane. Adjust the rotation matrices of the left and right cameras to:

[0058]

[0059] Among them, R rect is the rotation matrix used to rotate both cameras to the same reference frame.

[0060] (2) Through the new rotation matrix, the homography matrix can be calculated:

[0061]

[0062] Among them, K is the intrinsic parameter matrix of the camera.

[0063] (3) Apply the homography matrix to the original image coordinates to obtain the corrected image coordinates:

[0064]

[0065] Among them, p L and p R are the original points in the left image and the right image respectively, p ′ L and p ′ R are the corrected points respectively.

[0066] Step 1.2: Establish an anchor end image dataset and train an anchor end target detection algorithm:

[0067] Obtain image data containing anchor rod ends from the construction site or the Internet. Label the acquired image data and use professional labeling tools (such as LabelImg or Labelme) to manually label the location of the anchor rod ends. Select the anchor rod end area and set the label to "anchor_rod". The labeling results include the bounding box coordinates and category labels of the anchor rod ends in each image. Establish an anchor rod end image dataset.

[0068] The labeled image data is preprocessed, including image size standardization and data enhancement (such as rotation, scaling, flipping, etc.) to increase data diversity.

[0069] Select Yolov8 as the target detection algorithm, download and configure the Yolov8 model, including setting the network structure, selecting the pre-trained model, and configuring the training parameters: learning rate, batch size, number of training rounds, etc. Input the preprocessed image data into the Yolov8 model for training. During the training process, the model will learn based on the input image and the corresponding annotation file, and continuously adjust its internal parameters to improve the detection accuracy of the anchor end. During and after the training, use the validation set to validate the model and evaluate its detection performance. Based on the validation results, optimize and adjust the model. Finally, deploy the trained Yolov8 model to the anchor end recognition module.

[0070] Step 2: Use a binocular camera to collect images and perform image preprocessing:

[0071] Step 2.1, binocular camera image acquisition:

[0072] Ensure that the two cameras can shoot synchronously to obtain time-consistent image pairs. The captured images need to be transmitted to the image acquisition and preprocessing module in real time.

[0073] Step 2.2: Use adaptive median filter to remove image noise:

[0074] Adaptive median filter is used to remove dust interference and sudden noise in the mine environment. The calculation method is as follows:

[0075] (1) Define the filter window

[0076] Set a 3×3 window as the initial filter window, and the maximum filter window S max Defined as 9×9 size.

[0077] (2) Sorting pixel values ​​within the window

[0078] For the current window, extract all pixel values ​​within the window and arrange them in ascending order.

[0079] (3) Calculate the median, minimum, and maximum values

[0080] Let the sorted pixel values ​​in the window be {Z min ,Z med ,…,Z max}. Among them Z min ,Z med ,…,Z max are the minimum, median, and maximum values ​​within the window, respectively.

[0081] (4) Calculation phase A: Checking the median and maximum values

[0082] Calculate A1=Z respectively med -Z min and A2 = Z med -Z max And check the median and the maximum. Check the price adjustment as follows: If A1>0 and A2<0, then Z med If it is not noise, enter stage B. Otherwise, increase the window size and execute step 2 again; if Smax is reached, enter stage B directly.

[0083] (5) Calculation phase B: Check the current pixel value

[0084] Set the current pixel value to Z x , calculate B1 = Z x -Z min and B2 = Z x -Z max , and perform a conditional check: If B1>0 and B2<0, then Z x If it is not noise, keep the original value. Otherwise, use Z med Replace Z x The value of .

[0085] Step 2.3, setting the region of interest:

[0086] In a tunnel that has been supported, the position of the anchor is relatively fixed. Except for the sudden displacement of the anchor when a sudden disaster such as roof collapse occurs, the spatial position of the anchor end generally changes little. Therefore, the approximate location area of ​​each anchor end can be set in advance as the region of interest, and the subsequent target detection algorithm only searches within the region of interest.

[0087] According to the tunnel image collected by the camera, the approximate area of ​​the anchor end is manually marked. Assume that the pixel area of ​​the i-th anchor end in the image is {x, y, w, h} i , where (x i ,y i) is the coordinate of the center point of the anchor end area in the picture. i ,h i are the width and height of the anchor end area respectively. Take the center point of the anchor end as the center of the region of interest, set the side length of the region of interest to k, (k>w,h), then the range of the region of interest of this anchor end is {x i ,y i ,k,k} i .

[0088] Step 3: Use the pre-trained target detection algorithm to identify and locate the anchor end;

[0089] After image preprocessing, the tunnel images collected by the left and right cameras are passed to the anchor end recognition module for processing. The function of this module is to perform target detection, identify the anchor ends and generate the coordinate information of all anchor ends on the image. Set the algorithm confidence to 0.8, the number of iterations to 50, and the target detection bounding box to be drawn with a red line. The collected image is input into the trained target detection algorithm, and the algorithm searches in the area of ​​interest. After multiple iterations, the target detection algorithm will identify all anchor ends in the tunnel image and generate an identification bounding box around the anchor ends. The bounding box is a red rectangular box, and the position is determined by the center coordinates of the bounding box (x i ,y i ) and the width and height w,h of the frame. The module returns the recognition frame area of ​​all anchor ends. The recognition result of the image collected by the left camera is The image recognition result collected by the right camera is The subsequent anchor end coordinate solution will be carried out in the anchor end identification box.

[0090] Step 4: Calculate the three-dimensional coordinates of the anchor bolt end using binocular vision three-dimensional imaging technology based on the two-dimensional positioning information;

[0091] The three-dimensional coordinate solution of the bolt end mainly includes the steps of left and right camera image registration, three-dimensional coordinate solution of the bolt end, and coordinate system conversion. Feature matching is used to identify and match the same bolt end feature points in the binocular image for subsequent calculation of depth information. The three-dimensional coordinate solution uses the matched feature points to calculate the precise position of the bolt end in three-dimensional space. The coordinate system conversion converts the calculated three-dimensional coordinates into a unified coordinate system so that it can be aligned with the actual mine coordinate system for further analysis and application.

[0092] Step 4.1: Align the images collected by the left and right cameras

[0093] Step 4.1.1: Extract anchor rod end features in the target detection frame area:

[0094] The ORB (Oriented FAST and Rotated BRIEF) feature detection algorithm is used to find corner points or significant texture areas in the area around each anchor end identification box. ORB first uses FAST to detect key points, then calculates the Harris response value of each key point and selects the key point with the highest response. The direction is determined by the local grayscale change of each key point.

[0095] Step 4.1.2, feature descriptor calculation:

[0096] A descriptor is generated for each detected feature point, which is a binary encoding of the grayscale pattern in the neighborhood of the feature point. ORB uses the BRIEF descriptor, which generates a binary vector of length 256. ORB descriptor calculation relies on comparing the grayscale values ​​of the pixels around the feature point:

[0097] D=BRIEF(P M )(5)

[0098] Among them, P is the neighborhood pixel set of the feature point.

[0099] Step 4.1.3: Feature matching of images collected by left and right cameras:

[0100] Find the same feature point pairs by comparing the feature descriptors in the two images. Use a distance metric (such as the Hamming distance) to measure the similarity of the descriptors. The Hamming distance between descriptors is calculated as follows:

[0101]

[0102] Where: ⊕ represents the XOR operation, N is the length of the descriptor, which is generally 256 bits.

[0103] Step 4.1.4: Eliminate false matches:

[0104] The RANSAC (Random Sampling Consensus) algorithm removes false matches by repeatedly selecting subsets and fitting models to find the optimal model. It is based on the fact that most internal points should conform to the basic matrix constraints. Assuming that we use the basic matrix F to describe the geometric relationship between the two images, RANSAC tries to find an F that satisfies the most matching pairs:

[0105] x i ′T Fx i =0(7)

[0106] Among them, x i ′ and x i are the homogeneous coordinates of the matched feature point pairs.

[0107] Step 4.1.5, output the matching pair list:

[0108] The result of feature matching is a list containing matching pairs, each of which consists of the coordinates of two feature points: the coordinates of the feature point in the left image (x L ,y L ), the feature point coordinates in the right image (x R ,y R ), the matching pair can be expressed as Indicates that a feature point in the left image matches a feature point in the right image.

[0109] Step 4.2: Calculate the three-dimensional coordinates of the anchor end:

[0110] The matching pairs of corresponding feature points are obtained through image registration, and the disparity can be calculated through the matching pairs, so as to calculate the depth information of the object using the disparity.

[0111] Step 4.2.1, calculate the disparity:

[0112] Parallax is the horizontal distance difference between the corresponding feature points in the left and right images, which reflects the depth information of the object. The larger the parallax, the closer the object is to the camera. Through parallax, the depth in three-dimensional space can be calculated.

[0113] For each pair of matching feature points, calculate their horizontal (x-axis) difference:

[0114] d = x L -x R (8)

[0115] Step 4.2.2, depth calculation:

[0116] Depth calculation is based on parallax and camera parameters to determine the actual distance of objects in the scene. There is an inverse relationship between parallax and depth, and depth can be calculated from parallax through a formula.

[0117]

[0118] Where f is the focal length of the camera, which indicates the distance from the optical center of the camera to the projection plane; b is the baseline, which indicates the optical center O L and O R The distance between.

[0119] Step 4.2.3: Calculate the three-dimensional coordinates of the anchor end:

[0120] According to the depth value and the internal and external parameters of the camera, the pixels in the two-dimensional image can be converted into points in three-dimensional space. The formula for calculating the three-dimensional coordinates of the anchor end is as follows:

[0121]

[0122] Where (x, y) is the pixel coordinate in the image coordinate system. x ,c y _ is the principal point of the image (the intersection of the optical axis and the image plane). Since there are multiple feature points in the anchor end recognition frame for pairing, in order to facilitate the calculation of the anchor end displacement, the anchor end recognition frame can be regarded as a whole, and the average of the coordinates of multiple feature points is taken as the spatial coordinates of the anchor end, that is:

[0123]

[0124] in, is the spatial mean of the camera coordinate system of the mth anchor end.

[0125] Step 4.3: Coordinate system conversion

[0126] Convert the point in camera coordinates to a global or world coordinate system, consisting of a rotation matrix R obtained from camera calibration and a translation vector T. Use homogeneous coordinate representation to convert camera coordinates to world coordinates. Homogeneous coordinates introduce an extra dimension so that translation operations can be represented by matrix multiplication. Camera coordinates in homogeneous coordinate representation World Coordinates

[0127] Step 5, calculate the spatial position change of the anchor bolt end at different times, and use it to solve the displacement of the top and bottom plates and the displacement of the two sides in different areas of the tunnel;

[0128] Step 5.1: Calculation of spatial displacement of anchor bolt end:

[0129] Assume that time points t0 and t i The three-dimensional coordinates of the mth anchor end were collected respectively and The spatial displacement of the anchor end between these two moments is calculated using the Euclidean distance formula:

[0130]

[0131] in, is the mth anchor end at time t i - Spatial displacement within t0.

[0132] Step 5.2, calculation of tunnel deformation:

[0133] The overall stability and deformation of the tunnel are evaluated by monitoring the convergence of the roof and floor plates and the convergence of the two sides. The convergence of the roof and floor plates mainly observes the movement of the anchor ends located on the roof and the bottom plate markers. The convergence of the two sides mainly observes the movement of the anchor ends at the two sides of the tunnel.

[0134] Step 5.2.1: Calculation of top and bottom plate convergence:

[0135] In order to record the movement of the top and bottom plates in each area, several markers corresponding to the anchors of the top plate are arranged on the bottom plate, and the spatial distance between each top plate anchor and the bottom plate marker is recorded. At the initial state t0, the distance between the end of the mth anchor and the corresponding bottom plate marker is At the subsequent time point t i (i=1,2,…,n), the distance between the mth anchor end and the bottom plate marker is

[0136] Calculate the top and bottom plate convergence. For time point t i , the convergence calculation formula is: if It means the tunnel is shrinking and the top and bottom plates are getting closer. It indicates that the tunnel is expanding and the top and bottom plates are moving away.

[0137] Record the convergence values ​​of all areas of the tunnel roof and floor, and take the average to reflect the overall roof and floor convergence of the tunnel. The calculation formula is as follows:

[0138]

[0139] Where M is the number of all anchors in the tunnel roof.

[0140] Step 5.2.2, calculation of the convergence of the two gangs:

[0141] In order to record the deformation of the two sides of the tunnel, the distance between the anchor ends of the same horizontal elevation on the two sides of the tunnel is calculated. At the initial state t0, the distance between the kth anchor end on the left side of the tunnel and the corresponding anchor end on the right side of the tunnel is At the subsequent time point t i (i=1,2,…,n), the distance between the kth anchor end at the left side of the tunnel and the corresponding anchor end at the right side of the tunnel is

[0142] Calculate the convergence of the two sides of the lane. For time point t i , the convergence calculation formula is:

[0143]

[0144] if Indicates that the two sides of the lane are moving closer. It means that the two gangs in the alley should stay away from each other.

[0145] Record the convergence values ​​of the two sides of all areas in the lane, and take the average value to reflect the overall convergence of the two sides of the lane. The calculation formula is as follows:

[0146]

[0147] Among them, K is the number of all anchor rods on both sides of the tunnel.

[0148] See also Figure 6 This embodiment also provides a real-time lane deformation detection system based on binocular vision, including an image acquisition unit, an image preprocessing unit, an anchor end recognition unit, an anchor end three-dimensional coordinate solution unit, and a lane deformation solution unit. The above units together construct a complete image data processing and analysis process, from basic image acquisition and preprocessing to complex three-dimensional coordinate solution and deformation analysis, ensuring the high performance and accuracy of the system, and providing strong technical support for lane safety monitoring. Among them:

[0149] An image acquisition unit, used for acquiring and processing image data from a binocular camera in real time;

[0150] The image preprocessing unit uses an adaptive median filter to remove environmental noise and interference to ensure image quality; by setting the region of interest, the calculation amount of the subsequent target detection algorithm is reduced;

[0151] Anchor end recognition unit, used to identify and locate the anchor end in the image, ensuring high accuracy and high reliability of target detection, and providing precise two-dimensional coordinate data of the anchor end for subsequent analysis;

[0152] The three-dimensional coordinate solving unit of the anchor end is used to calculate the three-dimensional coordinates of the anchor end in the world coordinate system, and accurately solve the position in the three-dimensional space through feature point matching and parallax calculation;

[0153] The tunnel deformation calculation unit is used to analyze the changes in the three-dimensional coordinates of the anchor bolt end and calculate the movement of the top and bottom plates and the two sides of the tunnel.

[0154] The deployment of the detection system in the embodiment of the present invention takes a mine tunnel as an example, but the application of the present invention is not limited to underground mines. The system is also applicable to tunnel deformation monitoring of underground projects such as tunnels and subways.

[0155] Compared with the method of deformation detection of tunnels based on traditional sensors, the present invention has significant advantages. First, the system does not require preset monitoring points, and directly captures the entire tunnel through a binocular camera, providing higher accuracy and timeliness, while achieving a higher degree of automation. Secondly, compared with traditional deformation detection technology that relies on time-series distance measurement between point clouds, the present invention uses image recognition and stereo vision technology to perform fine characterization of the tunnel surface, which is more suitable for complex environments such as mine tunnels. Thirdly, the present invention uses deep learning algorithms, such as target detection models, to perform real-time automatic analysis of deformation data, which not only eliminates the subjective errors that may be caused by manual processing of measurement data, but also can accurately identify the specific location of the deformation in real time.

[0156] In addition, the equipment deployment of this system is simple and the maintenance cost is low. It is easy to deploy and use quickly in a mine environment, which greatly improves the efficiency and safety of underground inspections. In summary, the tunnel deformation detection technology based on binocular vision of the present invention realizes the ability to obtain the deformation position in the tunnel from end to end, greatly simplifies the workflow, and improves the detection efficiency and accuracy. Therefore, the present invention can not only realize real-time and accurate tunnel deformation monitoring, but also provide important technical support and decision-making basis for mine support and maintenance work.

[0157] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time detection method for lane deformation based on binocular vision, characterized in that: The following steps are involved: Step 1: Install and debug the binocular camera in the tunnel, and train the anchor end target detection algorithm; Step 2: Use a binocular camera to collect images and perform image preprocessing; Step 3: Use the pre-trained target detection algorithm to identify and locate the anchor end; Step 4: Calculate the three-dimensional coordinates of the anchor bolt end using binocular vision three-dimensional imaging technology based on the two-dimensional positioning information; Step 5: Calculate the spatial position changes of the anchor bolt ends at different times, and use this to solve the movement of the top and bottom plates and the two sides in different areas of the tunnel.

2. The method for real-time detection of lane deformation based on binocular vision according to claim 1, characterized in that: In step 1, the binocular camera is installed and debugged, and the anchor end target detection algorithm is trained, which specifically includes the following steps: Step 1.1: Install a binocular camera on the tunnel roof, and perform binocular camera calibration, distortion correction, and stereo correction debugging; Step 1.2: Establish an anchor end image dataset and train an anchor end target detection algorithm.

3. The method for real-time detection of lane deformation based on binocular vision according to claim 2, characterized in that: In step 2, a binocular camera is used to collect images and perform image preprocessing, which specifically includes the following steps: Step 2.1, using a binocular camera to collect images; Step 2.2, using an adaptive median filter to remove noise from the collected image; Step 2.3: Manually mark the approximate area of ​​the anchor end as the region of interest.

4. The method for real-time detection of lane deformation based on binocular vision according to claim 3, characterized in that: In step 4, based on the two-dimensional positioning information, the three-dimensional coordinates of the anchor end are calculated using binocular vision three-dimensional imaging technology, which specifically includes the following steps: Step 4.1, binocular camera image registration: including the steps of anchor rod end feature extraction in the target detection frame area, feature descriptor calculation, binocular camera image feature matching and output matching pair list; Step 4.2, Calculate the three-dimensional coordinates of the anchor end: obtain the matching pairs of corresponding feature points according to the image registration, calculate the disparity through the matching pairs, and then calculate the depth information of the object using the disparity; Step 4.3, coordinate system conversion: Convert the points in camera coordinates to a global or world coordinate system.

5. The method for real-time detection of lane deformation based on binocular vision according to claim 1, characterized in that: In step 5, the spatial position change of the anchor bolt end at different times is calculated, and the top and bottom plate movement and the two side movement in different areas of the tunnel are solved based on this, which specifically includes the following steps: Step 5.1, calculation of spatial displacement of the anchor bolt end: use the Euclidean distance formula to calculate the spatial displacement of the anchor bolt end at different times; Step 5.2, calculation of tunnel deformation: the overall stability and deformation of the tunnel are evaluated by monitoring the convergence of the top and bottom plates and the convergence of the two sides.

6. A real-time detection system for tunnel deformation based on binocular vision, characterized in that: include: An image acquisition unit, used for acquiring and processing image data from a binocular camera in real time; The image preprocessing unit uses an adaptive median filter to remove environmental noise and interference to ensure image quality; by setting the region of interest, the calculation amount of the subsequent target detection algorithm is reduced; Anchor end recognition unit, used to identify and locate the anchor end in the image, ensuring high accuracy and high reliability of target detection, and providing precise two-dimensional coordinate data of the anchor end for subsequent analysis; The three-dimensional coordinate solving unit of the anchor end is used to calculate the three-dimensional coordinates of the anchor end in the world coordinate system, and accurately solve the position in the three-dimensional space through feature point matching and parallax calculation; The tunnel deformation calculation unit is used to analyze the changes in the three-dimensional coordinates of the anchor bolt end and calculate the movement of the top and bottom plates and the two sides of the tunnel.

Images