Dynamic vision SLAM tunnel deformation detection method

Abstract

The invention discloses a dynamic visual SLAM tunnel deformation detection method, which is based on an ORBSLAM3 algorithm and is combined with a YOLOv10 target detection network to carry out dynamic target detection. Dynamic feature points are removed according to a detection result while the feature points are extracted; then, residual dynamic feature points and mismatched feature points are removed through epipolar geometric constraint; and finally, converting the generated sparse map into a dense point cloud map, and comparing the current data of the same position with the previous data by using a deformation detection algorithm, thereby completing accurate analysis of tunnel deformation. According to the method, the influence of the dynamic target can be effectively processed, and high-precision tunnel deformation detection is carried out, so that the accuracy and efficiency of deformation detection in the tunnel environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision processing, and particularly to a dynamic vision SLAM tunnel deformation detection method. Background Art

[0002] Tunnel deformation detection is a key research direction in the field of mobile robots, involving multiple complex technical aspects: object detection, three-dimensional map reconstruction, and deformation analysis, etc. In practical applications, how to accurately identify and analyze the changes in the tunnel structure in a dynamic environment is a major challenge. With the continuous progress of computer technology and the wide application of robot technology, mobile robots have great potential for inspection in complex environments such as tunnels, pipelines, and coal mine roadways, and can timely detect potential problems and avoid the occurrence of dangerous events.

[0003] Current mainstream technologies such as lidar based on Simultaneous Localization and Mapping (SLAM) are outstanding in generating high-precision maps, but their high costs limit their popularization in practical applications. In addition, the application of mechanical lidar in non-open environments still faces problems such as uneven ground, large vehicle vibrations, and many outliers in point cloud data, which will affect the accuracy and efficiency of map construction. Lidar usually provides two-dimensional data and can only obtain angular distance information in a plane. To obtain three-dimensional point cloud data, an additional coordinate system is required. Traditional tunnel deformation detection methods mostly rely on sparse feature point extraction and basic map reconstruction technologies, and are easily affected by mismatches and environmental changes when dealing with dynamic targets. These methods mainly focus on deformation detection in static environments and have relatively insufficient processing capabilities for dynamic changes and complex environments. This limitation makes it difficult to perform deformation analysis in real-time monitoring in dynamic environments. Summary of the Invention

[0004] In view of the above deficiencies of the prior art, the present invention provides a dynamic vision SLAM tunnel deformation detection method, which can effectively handle the influence of dynamic targets and perform high-precision tunnel deformation detection, thereby improving the accuracy and efficiency of deformation detection in tunnel environments.

[0005] To solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A dynamic vision SLAM tunnel deformation detection method includes the following steps:

[0007] S1. Obtain a YOLOv10 detection network model for dynamic object detection;

[0008] S2. Use an RGB-D camera to obtain an input image, getting an RGB image and the corresponding depth image. Take the RGB image and the corresponding depth image of the current frame as the input of the YOLOv10 detection network model and perform feature point extraction and calculation of descriptors corresponding to the feature points based on the ORB algorithm. After processing, background feature points and bounding boxes are obtained;

[0009] S3. Use the obtained background feature points and bounding boxes to detect the dynamic consistency between images, obtaining static feature points, dynamic feature points, and potential dynamic feature points, and removing the dynamic feature points;

[0010] S4. Match the feature points remaining after removing the dynamic feature points in the current frame with the feature points of the RGB image of the previous frame, and perform camera pose estimation based on the feature point matching results to obtain key frames. Then use the epipolar geometry constraint to remove the remaining dynamic feature points and mis-matched feature points in the potential dynamic feature points, obtaining carefully classified static feature points and dynamic feature points;

[0011] S5. Perform feature matching using the static feature points obtained in step S3 and the carefully classified static feature points in step S4, and combine with the key frames obtained in step S4 to generate a local map;

[0012] S6. According to the local map obtained in step S5, compare the similarity between the current frame and historical frames to perform loop detection;

[0013] S7. Based on the key frames obtained in step S4, use the key frames to perform RGB image and depth image matching. For each RGB pixel point, calculate the spatial coordinates to establish a dense point cloud map;

[0014] S8. Extract cross-sections from the dense point cloud map, randomly sample points on the extracted cross-sections, and compare the distances between the sample points with a preset distance threshold to obtain the tunnel deformation detection result.

[0015] Further, in step S2, before taking the RGB image as the input of the YOLOv10 detection network model for object detection, it further includes: performing proportional scaling and padding processing on the input RGB image according to the input size of the YOLOv10 detection network model. After normalizing the processed image, use it as the input of the YOLOv10 detection network model.

[0016] Further, the specific method of step S4 is:

[0017] S401. Construct an image pyramid model, perform downsampling and hierarchical processing on the RGB image;

[0018] S402. Calculate the Hamming distance between the descriptors of the remaining feature points and the feature points of the previous frame based on the processed RGB image, and select the pair of feature points with the smallest Hamming distance as the matching point pair;

[0019] S403. Sort the matched point pairs in ascending order according to the Hamming distance of the matched point pairs, and use the Random Sample Consensus (RANSAC) algorithm to obtain the fundamental matrix for the first 1 / 3 of the matched point pairs;

[0020] S404. Calculate the distance using the epipolar geometry constraint based on the obtained fundamental matrix. When the distance is greater than the preset threshold, the feature point is regarded as a dynamic feature point or a mismatched feature point and is removed; when the distance is less than or equal to the preset threshold, the feature point is a static feature point.

[0021] Further, in step S404, the formula for calculating the distance is expressed as:

[0022]

[0023] In the formula, d is the distance from P2 to the epipolar line defined by P1 and F, A, B, and C are all line vectors, P1 and P2 are a pair of matching points on two frames of RGB images, and F is the fundamental matrix.

[0024] Further, the specific method of step S6 is as follows:

[0025] Find the key frame that has a loop closure relationship with the current key frame in the list of historical key frames. If the detected loop closure key frame is in the local map, perform loop closure correction on the loop closure key frame; if the detected loop closure key frame is not in the local map, perform local map fusion on the loop closure key frame.

[0026] Further, the specific method of step S7 is as follows:

[0027] S701. Register the RGB image and the depth image of the current frame using the intrinsic and extrinsic parameters of the RGB-D camera;

[0028] S702. Convert the pixel points of the static feature points in the key frame from two-dimensional pixel coordinates to three-dimensional space coordinates;

[0029] S703. Combine the RGB image and the corresponding spatial coordinates obtained by the conversion to generate the point cloud of the current key frame;

[0030] S704. Fuse the point cloud of the historical key frame and the point cloud of the current key frame, and connect all the point clouds together according to the position information of the key frames obtained in ORB-SLAM3 to form a global point cloud map.

[0031] Further, in step S704, the formula for point cloud fusion is expressed as:

[0032] T = argmin T ∑ i ||R i P i + t i - Q i || 2 ;

[0033] In the formula, T is the point cloud fusion matrix, and argmin T represents finding the point cloud fusion matrix T that minimizes the subsequent expression. R i is the rotation matrix, P i is the three-dimensional coordinate of the i-th point in the source point cloud, Q i is the three-dimensional coordinate of the i-th point in the target point cloud, t i is the translation vector, and ||·|| is the Euclidean distance.

[0034] Furthermore, the specific method of step S8 is as follows:

[0035] S801. Determine the cross-section position in the tunnel, use a point cloud processing tool to cut the historical dense point cloud map model and the current dense point cloud map model, and extract the point cloud data on the preset cross-section;

[0036] S802. According to the extracted point cloud data on the preset cross-section, randomly sample points on the extracted cross-section model, and use a spatial search algorithm to find the corresponding matching point pairs in the historical dense point cloud map model and the current dense point cloud map model;

[0037] S803. Calculate the distance difference between each pair of matching point pairs in the historical dense point cloud map model and the current dense point cloud map model, and preset a distance threshold. When the distance difference of the matching point pair is greater than the preset distance threshold, mark the matching point pair as a dangerous point; if the distance difference of the matching point pair is less than or equal to the preset distance threshold, determine the matching point pair as a safe point;

[0038] S804. Repeat step S803, count the number of dangerous points on this cross-section, and compare it with the preset danger threshold. If the number of dangerous points on this cross-section exceeds the preset danger threshold, determine that the cross-section has deformed and execute step S805; if the number of dangerous points on this cross-section does not exceed the preset danger threshold, determine that the cross-section has not deformed, and complete the tunnel deformation detection;

[0039] S805. Repeat the above steps S801 to S804 to analyze multiple adjacent cross-sections of this cross-section. If the adjacent cross-sections have also deformed, confirm that the deformation of the cross-section in the tunnel is true; if the adjacent cross-sections have not deformed, determine that this cross-section has not deformed.

[0040] Further, in step S803, the formula for calculating the distance difference is expressed as:

[0041]

[0042] In the formula, is the distance difference between the historical dense point cloud map model and the current dense point cloud map model, is a sample point in the historical dense point cloud map model, is a sample point in the current dense point cloud map model.

[0043] Compared with the prior art, the present invention has the following technical effects:

[0044] (1) The present invention takes ORB_SLAM3 as the core, combines YOLOv10 to eliminate dynamic features and improve real-time performance, and uses epipolar geometric constraints to eliminate residual dynamic feature points and mismatched feature points, thereby eliminating the influence of environmental dynamic objects on the ORB_SLAM3 modeling; by constructing a dense point cloud map, tunnel deformation detection is further realized, enhancing the robustness and accuracy of the present invention in dynamic scenarios. Compared with traditional methods, the present invention not only improves the stability of positioning and mapping, but also effectively reduces the interference of dynamic objects on the system accuracy, ensuring reliability and accuracy in complex environments, thus significantly improving the effect and performance of tunnel deformation detection, and finally realizing the construction of a three-dimensional point cloud map at low cost to complete the reconstruction process of the mobile robot. By comparing the reconstruction results, the situation changes in the tunnel can be analyzed. Ensure the timely discovery of tunnel collapse and damage problems.

[0045] (2) Compared with traditional SLAM methods, this method uses the YOLOv10 algorithm. Through an efficient dual-task training strategy, it avoids the dependence on non-maximum suppression (NMS), thereby reducing the inference latency and having stronger end-to-end real-time detection capabilities; compared with other YOLO series, YOLOv10 based on the ORB_SLAM3 algorithm achieves an excellent balance between accuracy and efficiency at different model scales, and has higher versatility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the objectives, technical solutions and advantages of the invention clearer, the present invention will be further described in detail below with reference to the drawings, where:

[0047] Figure 1 is a flowchart of a method for detecting tunnel deformation by dynamic vision SLAM disclosed in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Components of the embodiments of the present invention described and illustrated herein can generally be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0049] The present invention will be further described in detail below with reference to the accompanying drawings.

[0050] As Figure 1 shown, the present invention discloses a dynamic vision SLAM tunnel deformation detection method, including the following steps:

[0051] S1. Obtain a YOLOv10 detection network model for dynamic target detection;

[0052] In this embodiment, an existing image dataset (such as the COCO dataset) can be selected to train the YOLOv10 network model, and the trained YOLOv10 model is obtained for detecting dynamic targets.

[0053] S2. Use an RGB-D camera to obtain an input image, obtain an RGB image and the corresponding depth image, use the RGB image and the corresponding depth image of the current frame as the input of the YOLOv10 detection network model, and perform feature point extraction and calculation of descriptors corresponding to the feature points based on the ORB algorithm, and obtain background feature points and bounding boxes after processing;

[0054] In this embodiment, the RGB-D image is scaled and padded proportionally according to the input size of the YOLOv10 detection network model, the input image is normalized to obtain I', and the input I' is input into the YOLOv10 detection network model. Feature extraction is performed on the input image through a convolutional neural network, and the output target result Y is expressed as:

[0055] Y = [b1, b2; …, b N ;

[0056] In the formula, Y is a three-dimensional tensor, is the prediction result of the i-th detection box, including four coordinates (x1, y1, x2, y2), a confidence score (score), and a class label (classID);

[0057] Through coordinate transformation, the coordinates output by the model are transformed from the input size to the original image size. YOLOv10 adopts a dual-label assignment and consistent matching metric method to achieve end-to-end processing. For regions where the object area density exceeds a given value, YOLOv10 avoids using non-maximum suppression (NMS) during the inference stage to improve real-time performance, obtaining bounding boxes and retaining the position information of the object contained in each bounding box, including the x and y coordinate values and the width and height. The formula for implementing end-to-end detection in YOLOv10 is expressed as:

[0058]

[0059] In the formula, m(a,β) is the metric function, s is the spatial prior of whether the predicted anchor is within the instance, p is the classification score, and b are the bounding boxes of the prediction and the instance respectively, and a and β are hyperparameters;

[0060] The image is divided into grids of a fixed size, and each grid cell predicts multiple bounding boxes and their class confidences. Each bounding box contains the confidence scores related to each class. Calculate the probability that each bounding box belongs to each class. Extract a set of target region sets B = {b1, b2, …, b n}, for each region, the model extracts the bounding box b i to represent the position of the object, extracts the feature vector v i of the target object and sends it to the subsequent network for inference, outputs the position of the detected target bounding box and clears the feature points extracted by ORB.

[0061] S3. Use the obtained background feature points and bounding boxes to detect the dynamic consistency between images, obtaining static feature points, dynamic feature points and potential dynamic feature points;

[0062] In this embodiment, by comparing the matching situation of the feature points in the current frame with the background feature points, the feature points with significant changes are identified as dynamic feature points, and at the same time, the feature points that remain consistent in multiple frames are retained as static feature points. Finally, in this way, the feature points are divided into static, dynamic and potential dynamic categories, realizing the effective distinction of the dynamic changes of objects in the image.

[0063] S4. Match the feature points remaining after removing the dynamic feature points in the current frame with the feature points of the RGB image of the previous frame, and perform camera pose estimation based on the feature point matching results, thereby obtaining key frames, and using the epipolar geometry constraint to remove the remaining dynamic feature points and mis-matched feature points in the potential dynamic feature points, obtaining the static feature points and dynamic feature points after detailed classification; Step S4 specifically includes:

[0064] S401. Build an image pyramid model to downsample and hierarchically process the RGB image;

[0065] In this embodiment, first of all, an image pyramid model needs to be constructed, which involves performing pyramid-style multi-level downsampling processing on the RGB image.

[0066] S402. According to the processed RGB image, calculate the Hamming distance between the remaining feature points and the descriptors of the feature points in the previous frame at the lowest level, and select the feature point pair with the smallest Hamming distance as the matching point pair;

[0067] Then, during the feature matching process, the Hamming distance can be calculated at each level of the image pyramid. However, since at high resolutions, the descriptors of the feature points can more accurately reflect the local features of the image, it is only necessary to calculate at the lowest level to ensure the accuracy of feature point matching.

[0068] S403. Arrange the matched point pairs in ascending order according to the Hamming distance of the matched point pairs, and use the Random Sample Consensus (RANSAC) algorithm to obtain the fundamental matrix for the first 1 / 3 of the matching point pairs;

[0069] Secondly, selecting the point pair with the smallest Hamming distance can ensure that these point pairs are the most similar in the feature space, which is helpful for improving the accuracy of fundamental matrix estimation. And by selecting the first 1 / 3 of the matching point pairs, while ensuring the accuracy of fundamental matrix estimation, the amount of calculation can be reduced and the computational efficiency of the algorithm can be improved.

[0070] S404. According to the obtained fundamental matrix, calculate the distance using the epipolar geometry constraint. When the distance is greater than the preset threshold, the feature point is regarded as a dynamic feature point or a mismatched feature point and is excluded; when the distance is less than or equal to the preset threshold, the feature point is a static feature point.

[0071] The formula for calculating the distance is expressed as:

[0072]

[0073] In the formula, d is the distance from P2 to the epipolar line defined by P1 and F, A, B, and C are all line vectors, P1 and P2 are a pair of matching points on two frames of RGB images, and F is the fundamental matrix.

[0074] S5. Use the static feature points obtained in step S3 and the static feature points carefully classified in step S4 for feature matching, and combine the key frames obtained in step S4 to generate a local map;

[0075] In this embodiment, point cloud data is extracted from static feature points, and a matching algorithm is used to pair the feature points in the current frame with those in the previous frame, which helps to establish a stable spatial correspondence relationship between different frames; the camera pose estimation technology is used to convert the three-dimensional coordinates of the static feature points into the global coordinate system to generate a local map, and local BA is applied to optimize the map, reduce errors and improve accuracy, and update and save the local map data.

[0076] S6. According to the local map obtained in step S5, compare the similarity between the current frame and the historical frame to perform loop closure detection;

[0077] In this embodiment, find the key frames that have a loop closure relationship with the current key frame in the historical key frame list. If the detected loop closure key frame is in the local map, perform loop closure correction on the loop closure key frame; if the detected loop closure key frame is not in the local map, perform local map fusion on the loop closure key frame.

[0078] This step performs historical frame loop closure detection to identify repeated positions, and applies loop closure graph optimization technology to improve the global Figure 1 consistency.

[0079] S7. Based on the key frames obtained in step S4, use the key frames to perform RGB image and depth image matching. For each RGB pixel point, calculate the spatial coordinates to establish a dense point cloud map; step S7 specifically includes:

[0080] S701. Use the internal and external parameters of the RGB-D camera to register the RGB image and the depth image of the current frame;

[0081] In this embodiment, register the RGB image and the depth map of the current frame to ensure that each RGB pixel point can accurately correspond to the depth value in the depth map, and use the internal and external parameters of the camera (such as the camera calibration matrix and distortion coefficient) for precise image correction to ensure the spatial position and scale consistency of the RGB image and the depth map.

[0082] S702. Convert the pixel points of the static feature points in the key frame from two-dimensional pixel coordinates to three-dimensional space coordinates;

[0083] In specific application embodiments, based on the pinhole camera model, the calculation formula for the three-dimensional space coordinates is:

[0084]

[0085] Z = d(u, v);

[0086] where u, v are the coordinates of the pixel point in the image, d is the depth value of the pixel point, c x and c yare the principal point coordinates of the camera, and f x and f y are the focal lengths of the camera, X and Y are the horizontal and vertical coordinates of the pixel point in the three-dimensional space respectively, and Z is the coordinate of the pixel point in the depth direction.

[0087] S703. Combine the RGB image and the converted corresponding spatial coordinates to generate the point cloud of the current key frame;

[0088] In this embodiment, the color information of the RGB image is combined with the calculated spatial coordinates to generate a point cloud with color information. The representation of each point cloud point is P=(X, Y, Z, R, G, B), and the point cloud data is denoised, sparsified, etc. to improve the quality and accuracy of the point cloud.

[0089] S704. Fuse the point cloud of the historical key frame and the point cloud of the current key frame, and connect all the point clouds together according to the position information of the key frames obtained in ORB-SLAM3 to form a global point cloud map.

[0090] In this embodiment, the point cloud data from different perspectives or frames is merged, and the ICP (Iterative Closest Point) algorithm or the voxel hashing method is applied to achieve the precise alignment and merging of multi-frame point clouds. The formula for point cloud fusion is expressed as:

[0091] T = argmin T ∑ i ||R i P i + t i - Q i || 2 ;

[0092] In the formula, T is the point cloud fusion matrix, and argmin T means to find the point cloud fusion matrix T that minimizes the subsequent expression. R i is the rotation matrix, P i is the three-dimensional coordinate of the i-th point in the source point cloud, Q i is the three-dimensional coordinate of the i-th point in the target point cloud, t i is the translation vector, and ||·|| is the Euclidean distance.

[0093] When generating a continuous and complete dense point cloud map, use a graph optimization algorithm such as BA (Bundle Adjustment) to optimize the entire point cloud map to reduce the cumulative error caused by sensor errors and drifts.

[0094] S8. Extract cross-sections from the dense point cloud map, randomly sample points on the extracted cross-sections, and compare the distances between the sample points with a preset distance threshold to obtain the tunnel deformation detection result.

[0095] According to the tunnel dense point cloud maps obtained at different time periods, they can be divided into a historical dense point cloud map model and a current dense point cloud map model. Step S8 specifically includes:

[0096] S801. Determine the cross-section position in the tunnel, use point cloud processing tools to cut the historical dense point cloud map model and the current dense point cloud map model, and extract the point cloud data on the preset cross-section;

[0097] In this embodiment, it is represented by setting a plane or a horizontal slice. Use point cloud processing tools to cut the historical model and the current model, extract the point cloud data on the defined cross-section, and then implement it through plane clipping or volume clipping techniques to limit the point cloud data to the specified cross-section.

[0098] S802. According to the extracted point cloud data on the preset cross-section, randomly sample points on the extracted cross-section model, and use a spatial search algorithm to find the corresponding matching point pairs in the historical dense point cloud map model and the current dense point cloud map model. The formula for calculating the distance difference is expressed as:

[0099]

[0100] In the formula, is the distance difference between the historical dense point cloud map model and the current dense point cloud map model, is the sample point in the historical dense point cloud map model, is the sample point in the current dense point cloud map model;

[0101] S803. Calculate the distance difference between each pair of matching point pairs in the historical dense point cloud map model and the current dense point cloud map model, and preset a distance threshold d1. When the distance difference of the matching point pair is greater than the preset distance threshold, that is then mark this matching point pair as a dangerous point; if the distance difference of the matching point pair is less than or equal to the preset distance threshold, then determine this matching point pair as a safe point;

[0102] S804. Repeat step S803, count the number N of dangerous points on this cross-section, and compare it with the preset danger threshold d2. If the number of dangerous points on this cross-section exceeds the preset danger threshold, that is N>d2, then determine that the cross-section has deformed, and execute step S805; if the number of dangerous points on this cross-section does not exceed the preset danger threshold, then determine that the cross-section has not deformed, and complete the tunnel deformation detection;

[0103] S805. Repeat the above steps S801 - S804 to analyze multiple adjacent cross - sections of this cross - section. If deformation also occurs on the adjacent cross - sections, it is confirmed that the deformation of the cross - section in the tunnel is true; if no deformation occurs on the adjacent cross - sections, it is determined that no deformation has occurred on this cross - section.

[0104] In this embodiment, if similar deformations also appear on the adjacent cross - sections, the deformations can be confirmed to be real; if only a certain cross - section shows deformation while the adjacent cross - sections have no similar changes, it may be an illusion caused by positioning errors. Finally, the analysis results of multiple cross - sections are integrated. If multiple cross - sections show consistent deformations, it is confirmed that these deformations actually exist.

[0105] In summary, the present invention takes ORB_SLAM3 as the core, combines YOLOv10 to eliminate dynamic features and improve real - time performance, uses epipolar geometry constraints to eliminate residual dynamic feature points and mismatched feature points, thereby eliminating the influence of environmental dynamic objects on the ORB_SLAM3 modeling; by constructing a dense point cloud map, it further realizes tunnel deformation detection, enhancing the robustness and accuracy of the present invention in dynamic scenarios. Compared with traditional methods, the present invention not only improves the stability of positioning and mapping, but also effectively reduces the interference of dynamic objects on the system accuracy, ensuring reliability and accuracy in complex environments, thus significantly improving the effect and performance of tunnel deformation detection, and finally realizing the construction of a three - dimensional point cloud map at low cost to complete the reconstruction process of the mobile robot. By comparing the reconstruction results, the situation changes in the tunnel can be analyzed. It ensures the timely discovery of tunnel collapse and damage problems.

[0106] Compared with traditional SLAM methods, this method uses the YOLOv10 algorithm. Through an efficient dual - task training strategy, it avoids relying on non - maximum suppression (NMS), thereby reducing the inference latency and having stronger end - to - end real - time detection capabilities; compared with other YOLO series, YOLOv10 based on the ORB_SLAM3 algorithm achieves an excellent balance between accuracy and efficiency at different model scales, with higher versatility and adaptability.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described by referring to the preferred embodiments of the present invention, those of ordinary skill in the art should understand that various changes can be made in form and details without departing from the spirit and scope of the present invention defined by the appended claims.

Claims

1. A dynamic visual SLAM tunnel deformation detection method, characterized in that: The steps include: S1. Obtain a YOLOv10 detection network model for dynamic target detection; S2. Use an RGB-D camera to obtain an input image, obtain an RGB image and a corresponding depth image, use the RGB image of the current frame and the corresponding depth image as the input of the YOLOv10 detection network model, extract feature points based on the ORB algorithm, and calculate the descriptors corresponding to the feature points. After processing, background feature points and bounding boxes are obtained; S3, using the obtained background feature points and bounding boxes to detect dynamic consistency between images, obtain static feature points, dynamic feature points and potential dynamic feature points, and remove the dynamic feature points; S4, matching the feature points remaining after removing the dynamic feature points in the current frame with the feature points of the RGB image of the previous frame, and performing camera pose estimation based on the feature point matching results, thereby obtaining a key frame, and using epipolar geometry constraints to remove the residual dynamic feature points and mismatched feature points in the potential dynamic feature points, to obtain carefully classified static feature points and dynamic feature points; S5, using the static feature points obtained in step S3 and the static feature points carefully classified in step S4 to perform feature matching, and combining the key frames obtained in step S4 to generate a local map; S6, comparing the similarity between the current frame and the historical frame according to the local map obtained in step S5, and performing loop detection; S7, based on the key frame obtained in step S4, use the key frame to match the RGB image with the depth image, calculate the spatial coordinates for each frame of RGB pixel points, and establish a dense point cloud map; S8. Extract the cross section from the dense point cloud map, randomly select sample points on the extracted cross section, and compare the distance between the sample points with the preset distance threshold to obtain the tunnel deformation detection result.

2. The dynamic vision SLAM tunnel deformation detection method according to claim 1, characterized in that: In step S2, before using the RGB image as the input of the YOLOv10 detection network model for object detection, the method also includes: scaling and padding the RGB image as the input according to the input size of the YOLOv10 detection network model, normalizing the processed image, and using it as the input of the YOLOv10 detection network model.

3. The dynamic vision SLAM tunnel deformation detection method according to claim 1, characterized in that, The specific method of step S4 is: S401, constructing an image pyramid model, downsampling and hierarchical processing of RGB images; S402, calculating the Hamming distance between the remaining feature points and the descriptors of the feature points of the previous frame according to the processed RGB image, and selecting the feature point pair with the smallest Hamming distance as the matching point pair; S403, arranging the matched point pairs in ascending order according to the Hamming distances of the matched point pairs, and obtaining a basic matrix for the first 1 / 3 of the matched point pairs using a random sampling consensus algorithm; S404, calculating the distance using the epipolar geometry constraint according to the obtained basic matrix, and when the distance is greater than a preset threshold, the feature point is regarded as a dynamic feature point or a mismatched feature point and is removed; When the distance is less than or equal to the preset threshold, the feature point is a static feature point.

4. The dynamic vision SLAM tunnel deformation detection method according to claim 3, characterized in that, In step S404, the formula for calculating the distance is expressed as: Where d is the distance from P2 to the epipolar line defined by P1 and F, A, B, and C are all line vectors, P1 and P2 are a pair of matching points on two RGB images, and F is the basic matrix.

5. The dynamic vision SLAM tunnel deformation detection method according to claim 1, characterized in that: The specific method of step S6 is: Find the keyframe that has a closed-loop relationship with the current keyframe in the historical keyframe list. If the detected closed-loop keyframe is in the local map, perform loop correction on the closed-loop keyframe; if the detected closed-loop keyframe is not in the local map, perform local map fusion on the closed-loop keyframe.

6. The dynamic vision SLAM tunnel deformation detection method according to claim 1, characterized in that: The specific method of step S7 is: S701, registering the RGB image and depth image of the current frame using the internal and external parameters of the RGB-D camera; S702, converting the pixel points of the static feature points in the key frame from two-dimensional pixel coordinates to three-dimensional space coordinates; S703, combining the RGB image and the corresponding spatial coordinates obtained by conversion to generate a point cloud of the current key frame; S704: Fuse the point cloud of the historical key frame with the point cloud of the current key frame, and connect all the point clouds together according to the position information of the key frame obtained in ORB-SLAM3 to form a global point cloud map.

7. The dynamic vision SLAM tunnel deformation detection method according to claim 6, characterized in that: In step S704, the formula for point cloud fusion is expressed as: T=argmin T ∑ i ||R i P i +t i -Q i || 2 ; Where T is the point cloud fusion matrix, argmin T Represents finding the point cloud fusion matrix T, R that minimizes the subsequent expressions i is the rotation matrix, P i is the 3D coordinate of the i-th point in the source point cloud, Q i is the 3D coordinate of the i-th point in the target point cloud, t i is the translation vector, and ||·|| is the Euclidean distance.

8. The dynamic vision SLAM tunnel deformation detection method according to claim 1, characterized in that: The specific method of step S8 is: S801, determining the cross-section position in the tunnel, using a point cloud processing tool to cut the historical dense point cloud map model and the current dense point cloud map model, and extracting point cloud data on the preset cross-section; S802, randomly extracting sample points on the extracted cross-section model according to the extracted point cloud data on the preset cross-section, and using a spatial search algorithm to find corresponding matching point pairs in the historical dense point cloud map model and the current dense point cloud map model; S803, calculating the distance difference between each pair of matching point pairs in the historical dense point cloud map model and the current dense point cloud map model, and presetting a distance threshold, when the distance difference of the matching point pair is greater than the preset distance threshold, marking the matching point pair as a dangerous point; if the distance difference of the matching point pair is less than or equal to the preset distance threshold, determining the matching point pair as a safe point; S804, repeat step S803, count the number of dangerous points on the section, and compare it with the preset danger threshold. If the number of dangerous points on the section exceeds the preset danger threshold, it is determined that the section has been deformed, and step S805 is executed; if the number of dangerous points on the section does not exceed the preset danger threshold, it is determined that the section has not been deformed, and the tunnel deformation detection is completed; S805, repeating the above steps S801 to S804, analyzing multiple adjacent sections of the section, and if deformation also occurs on the adjacent sections, it is confirmed that the section in the tunnel is indeed deformed; If the adjacent section has not been deformed, it is determined that the section has not been deformed.

9. The dynamic vision SLAM tunnel deformation detection method according to claim 8, characterized in that: In step S803, the formula for calculating the distance difference is expressed as: In the formula, is the distance difference between the historical dense point cloud map model and the current dense point cloud map model, are sample points in the historical dense point cloud map model, is the sample point in the current dense point cloud map model.

Images