A tunnel face lithology analysis method based on fusion of images and 3D point clouds

Abstract

The present invention relates to a method for analyzing the lithology of a tunnel face by fusing images and three-dimensional point clouds. First, a binocular camera is used to obtain a high-quality image of the face and generate a three-dimensional point cloud model. Next, the image and point cloud data are preprocessed, and the structural surface traces of the face are extracted using advanced algorithms. On this basis, a deep learning model is trained in combination with expert annotation data to achieve automatic segmentation and recognition of traces. By optimizing and fusing the model, the two-dimensional and three-dimensional joint features of the face are further extracted, and the lithology classification index is calculated. This method can significantly improve the accuracy and efficiency of lithology analysis, and provides important support for technological progress in the field of tunnel engineering construction. This lithology analysis method that combines image and three-dimensional point cloud technology not only improves the degree of automation of data processing, but also reduces the interference of human factors, and has broad application prospects.

Description

Technical Field

[0001] The present invention relates to the fields of rock tunnel engineering technology, computer vision and deep learning, and in particular to a method for analyzing the lithology of a tunnel face by fusing images and three-dimensional point cloud data. Background Art

[0002] In the field of tunnel engineering construction, accurate analysis of the rock properties of the tunnel face is a key link in ensuring construction safety, improving engineering quality, and optimizing construction plans. Traditional rock property analysis methods mainly rely on engineers' visual observation and empirical judgment, which is not only inefficient but also easily affected by human factors, resulting in inaccurate and unreliable analysis results. In addition, for complex geological conditions, such as interlaced rock layers and developed fault zones, traditional methods often have difficulty in accurately judging rock properties and their distribution.

[0003] With the continuous development of science and technology, image processing technology and 3D point cloud technology are gradually applied to the lithology analysis of tunnel faces. Image processing technology can extract the texture, color, shape and other characteristics related to lithology by digital processing and analysis of the face image. These characteristics reflect the physical and chemical properties of the rock to a certain extent, and provide important information for lithology analysis. At the same time, 3D point cloud technology can construct a high-precision 3D model of the face by acquiring 3D point cloud data of the face through laser scanning or other means. This model not only contains the geometric morphology information of the face, but also reflects the detailed characteristics of the rock such as undulations and cracks. Through the analysis of the 3D model, we can have a more comprehensive and in-depth understanding of the lithology distribution and geological structure of the face.

[0004] However, single image processing or 3D point cloud technology still has some limitations in the analysis of lithology of tunnel faces. Although image processing can extract rich 2D features, it is difficult to obtain 3D morphological information of the tunnel face; and although 3D point cloud technology can construct a 3D model of the tunnel face, it is still insufficient for detailed analysis of lithology. In addition, the data processing and analysis methods of these two technologies also have certain differences and limitations, making it difficult to give full play to their respective advantages.

[0005] Therefore, how to effectively integrate image processing and 3D point cloud technology to achieve a comprehensive and accurate analysis of the lithology of the tunnel face has become a hot topic and difficulty in current research. By integrating these two technologies, we can make full use of their respective advantages, make up for each other's shortcomings, and improve the accuracy and reliability of lithology analysis. At the same time, with the continuous development of artificial intelligence technologies such as machine learning and deep learning, this integration has also provided more possibilities and application prospects.

[0006] In the prior art, patent applications such as "Rock Tunnel Face Analysis Feedback Integrated System Based on 3D Laser Scanning" with publication number CN1100260785A, "Tunnel Face Analysis Method Based on Image Data" with publication number CN112215820A, and "Rock Lithology Analysis Method, Device and Equipment Based on Visualization Algorithm" with publication number CN112926602A all discuss the use of single image data or 3D point cloud data for lithology identification. However, the main deficiencies of these technologies are:

[0007] First, for single image data, its shortcomings are mainly manifested in the following aspects: ① Limited information dimension: Image data mainly provides two-dimensional plane information, which is difficult to accurately reflect the three-dimensional morphology and geological structure of the tunnel face. This limitation of information dimension may lead to insufficient accuracy in lithology identification. ② Influence of light and shadow: Image data is easily affected by light and shadow, resulting in unstable image quality. This instability will further affect the accuracy of lithology identification.

[0008] Then, for single 3D point cloud data, its shortcomings are mainly manifested in the following aspects: ① Large data volume and complex processing: 3D point cloud data usually contains a large amount of spatial point information, which is relatively complex to process. This not only increases the difficulty of data processing, but also may affect the efficiency of lithology identification. ② High requirements for hardware equipment: Acquiring and processing 3D point cloud data requires high-precision hardware equipment, such as laser scanners. These devices are expensive, and the operation and maintenance are relatively complex.

[0009] In summary, the present invention proposes a tunnel face lithology analysis method that integrates images and 3D point clouds, aiming to achieve rapid and accurate analysis of the lithology of the tunnel face by combining the advantages of these two technologies. This will not only help improve the safety and quality of tunnel engineering, but also help promote scientific and technological progress and development in the field of tunnel engineering construction. Summary of the invention

[0010] This paper proposes a tunnel face lithology analysis method that integrates images and 3D point clouds, aiming to achieve a comprehensive, accurate and efficient analysis of the lithology of the tunnel face by combining the advantages of image processing and 3D point cloud technology. Traditional lithology analysis methods mainly rely on engineers' visual observation and empirical judgment, which is not only inefficient but also easily affected by human factors, resulting in inaccurate and unreliable analysis results.

[0011] In order to solve the above problems, the present invention adopts the following technical solutions:

[0012] Firstly, a binocular camera is used to collect high-quality image data of the tunnel face, and a three-dimensional point cloud model of the tunnel face is generated through three-dimensional reconstruction technology.

[0013] Next, the image data is preprocessed, including image correction, noise removal, image sharpening, edge detection, and image registration, to extract the structural traces of the face. At the same time, the 3D point cloud data is preprocessed by noise reduction, enhancement, registration, and alignment to obtain high-quality point cloud data.

[0014] In the feature extraction stage, the structural surface traces of the tunnel face are extracted by combining the 2D image data and 3D point cloud data using advanced algorithms and technologies. The extracted structural surface trace images and point cloud data are fused and analyzed, and combined with expert human intervention and modification to form a labeling database.

[0015] Subsequently, the deep learning model is trained using the annotated database. By inputting the pre-processed image data and point cloud data, as well as the corresponding annotated data, the deep learning model is trained to achieve automatic segmentation and recognition of the tunnel face traces. By optimizing the model structure and parameter settings, the recognition accuracy and generalization ability of the model are improved.

[0016] Finally, the results of the deep learning model output are integrated and analyzed to extract the two-dimensional joint data and three-dimensional joint features, and the lithology classification index is calculated to complete the lithology classification. At the same time, the results of the deep learning model output are corrected by expert intervention and added to the annotation database for cyclic learning. Through continuous iterative optimization, the learning and analysis capabilities of the deep learning model are further improved.

[0017] The deep learning model used in this invention is BASeg, which is improved based on UniSeg:

[0018] The feature extraction head is mainly built using 2D convolutional neural networks and 3D convolutional neural networks, which can simultaneously extract features of image data and point cloud data. The boundary attention module is also integrated to enhance the model's attention to boundary area features.

[0019] The backbone adds a voxel-image cross-modal association module VI, which is responsible for automatically fusing features from different views with image features, utilizing the rich semantic information in the image, and enhancing robustness to calibration errors. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is the overall flow chart of the present invention;

[0021] Figure 2 A flowchart of a neural network training process provided by an embodiment of the present invention;

[0022] Figure 3 Schematic diagram of the neural network model (feature extraction head) for image and point cloud fusion extraction used in this method;

[0023] Figure 4 Schematic diagram of the BAI module for edge attention mechanism processing two-dimensional images;

[0024] Figure 5 Schematic diagram of the BAP module for edge attention mechanism processing 3D point cloud;

[0025] Figure 6 Schematic diagram of the voxel image cross-modal association VI module. DETAILED DESCRIPTION

[0026] In order to have a deeper understanding of the technical solution proposed by the present invention, the specific implementation cases of the present invention will be described in detail below, and the drawings will be used for intuitive explanation. It should be emphasized that any other implementation methods that can be derived or realized by any ordinary technician in the field without creative work should be deemed to fall within the protection scope of the present invention.

[0027] An embodiment of the present invention provides a tunnel face lithology analysis method that fuses images and three-dimensional point clouds. The method divides the tunnel face lithology analysis process into three parts: traditional method extraction, deep learning method extraction, and cyclic learning fusion.

[0028] First, the processed 2D image data and 3D point cloud data are processed according to traditional methods to extract joint data, and then experts intervene and modify them, and the data are fused and analyzed to form a labeled database;

[0029] Then, the preprocessed data is used as the input of the deep learning model, and the data in the annotated database is used as the label of the deep learning model for training;

[0030] Finally, the results output by the deep learning model are fused and analyzed, and the two-dimensional joint data and three-dimensional joint characteristics are extracted to calculate the lithology classification index to complete the lithology classification. At the same time, the results output by deep learning are corrected by expert human intervention and added to the annotation database for cyclic learning.

[0031] refer to Figure 1 to Figure 5 , the specific steps of this method are as follows:

[0032] Step 1. Perform manual discrimination and screening on the two-dimensional tunnel face images collected by the binocular camera, and perform three-dimensional reconstruction on the tunnel face images with relatively high quality to obtain a three-dimensional point cloud model of the tunnel face.

[0033] Step 2. Perform image correction, noise removal, image sharpening, edge detection and image registration on the selected two-dimensional tunnel face image to extract the tunnel face structure trace image.

[0034] The main methods used for image correction are histogram equalization and gamma correction. The main purpose of gamma correction is to adjust the contrast of the image. The larger the gamma value, the clearer the dark details; the smaller the gamma value, the clearer the bright details;

[0035] The main method of noise removal is median filtering or Gaussian filtering, which is mainly used to reduce random noise in the image and improve clarity;

[0036] The main methods used for image sharpening are Laplacian operator and unsharp masking to enhance the edge of the image and facilitate the extraction of two-dimensional joint features;

[0037] Finally, image registration is performed, mainly using feature matching and image refinement to extract the tunnel face structure trace image, i.e., the two-dimensional image data of the tunnel face structure trace.

[0038] Step 3. Perform preprocessing operations such as point cloud denoising, point cloud enhancement, point cloud registration and alignment on the three-dimensional point cloud data of the tunnel face; the point cloud data denoising methods include mean filtering, median filtering, Gaussian smoothing filtering and point cloud resampling, and these methods are used to reduce the noise of the point cloud data; at the same time, normal estimation and scale adjustment are used to enhance the point cloud;

[0039] Then, the ICP (Iterative Closest Point) and NDT (Normal Distribution Transform) algorithms are used to register and align the point cloud to complete the preprocessing of the tunnel face point cloud data.

[0040] Step 4. Perform feature extraction on the preprocessed 3D point cloud data to obtain the structure trace of the tunnel face.

[0041] Feature extraction mainly follows the following steps:

[0042] First, the triangulated network is established using the Delaunay triangulation algorithm;

[0043] Then, the normal tensor voting (NTV) method is used to detect feature points and detect the crack feature points of the tunnel face in the point cloud data.

[0044] Then, a point cloud compression algorithm based on the Laplacian operator is used to shrink the feature points;

[0045] Next, the feature points are grouped and connected;

[0046] Finally, the feature points are linearized and the linearized trajectory segments are connected to extract the point cloud data of the tunnel face structure trace, that is, the three-dimensional point cloud data.

[0047] Step 5. The extracted 2D image data and 3D point cloud data are fused and analyzed, and modified through expert intervention to form a labeling database, specifically:

[0048] The alignment of the tunnel face structural trace image with the point cloud data, that is, matching the points on the two-dimensional image with the points in the three-dimensional space, is achieved through feature point matching and geographic coordinate conversion. Experts identify the key features on the tunnel face (such as cracks, joints, lithology changes, etc.) based on the fused data; then annotate these key features on the two-dimensional image and three-dimensional point cloud data (including marking positions, measuring dimensions, recording attribute information, etc.). Experts correct and supplement possible errors or omissions in automatic identification or preliminary annotation to ensure the accuracy and completeness of the annotation. Finally, the annotated image and point cloud data and related attribute information are stored in the database for subsequent query and analysis.

[0049] Step 6. Use the preprocessed two-dimensional image data and three-dimensional point cloud data as the input of the deep learning model, and use the data in the annotation database as the label of the deep learning model to perform model training. The output of the deep learning model is the automatic segmentation and recognition of the track line of the tunnel face. The image data is input into the deep learning model. The deep learning model used in the present invention is the BASeg model improved based on UniSeg.

[0050] The main steps of model training are:

[0051] First, after processing the data, the traces extracted using traditional methods are used as “Ground truth”;

[0052] Then, the data is divided into training set, validation set, and test set.

[0053] Then, perform model training evaluation, assessment and optimization:

[0054] Use the training set to train the model, calculate the output through forward propagation, and then update the weights through the back-propagation algorithm; after each epoch, use the validation set to evaluate the model performance to ensure that the model is not overfitting; optimize the model by adjusting the number of convolutional layers, using different activation functions, and adjusting the network depth; evaluate the model based on the performance of the validation set on the model (accuracy, recall rate, F1 score, etc.), continuously optimize, and finally save the best weights for trace extraction.

[0055] Based on the results of segmentation and feature extraction, Hough transform is used to generate joint traces from the point cloud.

[0056] The generated traces are optimized (including removing short or discontinuous trace segments), and adjacent traces are connected to form complete jointed traces.

[0057] Finally, the extracted trace characteristics, trace continuity, direction, length, etc. are analyzed to provide data support for the stability analysis of tunnel engineering.

[0058] Step 7. Fusion analysis is performed on the output results of the deep learning model to extract two-dimensional joint data and three-dimensional joint features, calculate lithology classification indicators, and complete lithology classification;

[0059] First, the results of data processing by the deep learning model (BASeg) are integrated to ensure that the joint features identified in the 2D image correspond to the corresponding features in the 3D point cloud. The texture and color of the joints are extracted from the 2D image, and the spatial position, depth, width and other features of the joints are extracted from the 3D point cloud.

[0060] Step 8. Experts intervene and correct the results of deep learning output, add them to the annotation database, and conduct cyclic learning to further improve the learning ability of the model.

[0061] The output of the deep learning model is mainly reviewed by geological experts to identify possible errors or omissions, and the joint features that the model failed to accurately identify or misclassified are manually corrected. The corrected data and the expert annotations are then updated to the annotation database. The deep learning model is retrained using the updated annotation database to improve the accuracy and generalization ability of the model. Finally, the model parameters and structure are adjusted and optimized based on the expert feedback and model performance.

[0062] In step 6, BASeg plays a key role in extracting the tunnel face traces from the image data. Figure 3 ,BASeg model mainly includes the following modules:

[0063] Module 1: Feature Extraction

[0064] The two-dimensional convolutional network is used to extract image features from the face image data. As the front-end part of the BASeg model, ResNet is used to process the input two-dimensional image data.

[0065] The face point cloud data first goes through a voxelization process to discretize the continuous three-dimensional space into voxels (voxels are pixels in three-dimensional space). This step is used to convert the face point cloud data into a format that can be processed by the neural network. The voxel encoder is then used to encode the voxelized data and extract voxel features in preparation for subsequent processing.

[0066] Module 2: Boundary Attention Module (BA)

[0067] The main function of this module is to extract edge semantics related to the trace joint boundary from low-level features, which helps the model identify the local edge details and global position information of the joint.

[0068] refer to Figure 4 ,The BAI module of edge attention mechanism for processing two-dimensional images mainly consists of several parts:

[0069] First, the data will be extracted into multi-level features F1 and F2 when it passes through the backbone network feature extraction;

[0070] Then, two 1×1 convolutional layers are used to change the F1 and F2 channels to 64 (F1′) and 256 (F2′), respectively.

[0071] Then, the feature F1′ and the upsampled F2′ are concatenated through a cascade operation.

[0072] Finally, after being processed by two 3×3 convolutional layers and one 1×1 convolutional layer, the sigmoid function is used to obtain the image feature FI.

[0073] refer to Figure 5 ,The BAP module of edge attention mechanism for processing 3D point cloud mainly consists of several parts:

[0074] First, after voxelization, the point cloud data will be extracted into multi-level voxel features F1 and F2.

[0075] Then, two 1×1×1 convolutional layers are used to change the channels of F1 and F2 to 64 (F1′) and 256 (F2′), respectively.

[0076] Then, the converted voxel feature F1′ and the upsampled F2′ are concatenated through a cascade operation.

[0077] Finally, after processing through three 3×3×3 convolutional layers and one 1×1×1 convolutional layer, the voxel feature FV is obtained using the sigmoid function.

[0078] Experiments have shown that this module is a simple and effective module for extracting specific edge features.

[0079] Module 3: Voxel Image Cross-Modal Association Module (VI)

[0080] like Figure 6 As shown in Figure 1, this module fuses the image features FI and the voxelized point cloud features FV.

[0081] After the boundary features are refined and extracted by the boundary attention module, in this module: first, the image features corresponding to the voxel center are obtained, and then the image features are sampled using the learned offset. The voxel features are processed as queries, and the sampled image features are represented as keys and values. The voxel and sampled image features are sent to the multi-head cross attention module to obtain the voxel features of the image enhancement. These features are spliced ​​with the original features to obtain the final fused features.

[0082] Then, the fused features are input into a multi-layer perceptron to output the final segmentation result (which includes the segmented trace texture).

[0083] Finally, the segmentation results are fused and matched, and the results of the image data are used to enrich the results of the point cloud data. This part is manually intervened by experts to manually compare and correct the final segmentation results to make the segmentation results more accurate and corresponding Figure 1 The output results of the deep learning model are "humanly corrected by geological experts."

[0084] Step 7. Calculate the lithology classification index based on the trace and joint features extracted from the image and point cloud.

[0085] The main methods and indicators used are as follows:

[0086] BQ method:

[0087] The basic rock mass quality index BQ is expressed as follows:

[0088] BQ=100+3R c +250K v

[0089] When R c >90K v +30, with R c =90K v +30 is substituted into the above formula to calculate the BQ value; when K v >0.04R c +0.4, K v =0.04R c +0.4 is substituted into the above formula to calculate the BQ value. c is the saturated uniaxial compressive strength of rock (MPa); K v is the integrity coefficient of the rock mass. It can be determined using the acoustic wave test data as follows:

[0090]

[0091] Among them, ν pm is the rock mass longitudinal wave velocity, ν pr is the rock mass longitudinal wave velocity

[0092]

[0093] Q system method:

[0094] The calculation formula of Q system method is:

[0095]

[0096] Among them, RQD (Rock Quality Designation) is a rock quality design index, which is an indicator of rock integrity obtained through core drilling analysis, and its value range is 0-100.

[0097] J n is the joint number, which indicates the density and distribution of joints in the rock.

[0098] J r Joint roughness describes the roughness of rock joint surfaces.

[0099] J a It is a joint corrosion score, which indicates the degree of corrosion and deterioration of rock joint surfaces.

[0100] J w is the water pressure coefficient, which represents the influence of groundwater on the stability of rock mass.

[0101] SRF is the stress reduction factor, which indicates the influence of ground stress on the stability of rock mass.

[0102] The Q value generally ranges from 0.001 to 1000. The larger the Q value, the better the quality of the rock mass, and vice versa. Based on this, the rock mass quality is divided into 9 quality grades.

[0103]

[0104] GSI Method:

[0105] The GSI method is a method proposed by Hoek based on the Hoek-Brown strength criterion that reflects the nonlinear strength characteristics of rock. The essence of this evaluation is to modify the Hoek-Brown strength criterion in the presence of joints. Therefore, the results obtained by the GSI method can be directly linked to the mechanical properties of the rock mass. Compared with the evaluation results based on empirical criteria such as BQ and RMR, this method has stronger theoretical support.

[0106] RQD=110-2.5J v

[0107] J v --Number of rock mass volume joints

[0108] GSI=1.5Jcond 89+RQD / 2

[0109] Jcond 89 The values ​​are as follows

[0110]

[0111] Specifically in this embodiment, Figure 1 The flowchart of the tunnel face lithology analysis method provided in this embodiment includes the following steps:

[0112] Data collection and preparation In order to better conduct experiments, the data set is composed of the collected tunnel face pictures and the corresponding tunnel face point cloud data. After layers of screening, the tunnel face data with high quality suitable for the role was selected. Then the data was preprocessed, including image denoising, contrast adjustment, point cloud filtering and resampling, etc., to improve data quality. At the same time, image rotation, scaling, flipping and other technologies, as well as random sampling of point clouds, were applied to increase data diversity.

[0113] Traditional method for trace extraction According to the requirements of the above steps 1 to 5, the traditional method is used to extract the tunnel face traces and perform fusion analysis on the three-dimensional joint features. Experts intervene and modify the fusion results to form a complete annotated data set.

[0114] Deep learning method for trace extraction.

[0115] Model construction. According to the requirements of the previous step 6, this example uses the Python programming language to build the BASeg model. The Pytorch framework is used in the development of the BASeg model. The network structure including convolutional layers, pooling layers, and fully connected layers is designed, and the boundary attention module and feature fusion module are integrated into the network.

[0116] Model training. Through multiple iterations of forward propagation and back propagation on the training data, the model can automatically adjust its parameters according to the patterns and regularities in the data. This process usually includes multiple iterations, each of which contains two key steps: forward propagation and back propagation.

[0117] In the forward propagation phase, the model calculates the input data according to the current parameter settings and produces the corresponding output. These outputs are then compared with the true labels or expected outputs in the training data to calculate the loss value (that is, the difference between the predicted value and the true value).

[0118] Next is the back-propagation phase, which is the key to model optimization. In this phase, based on the loss value, an optimization algorithm such as gradient descent is used to calculate how to adjust the model's parameters to minimize the loss value. These parameter adjustments are small, but after many iterations, they will accumulate into significant improvements, gradually enhancing the model's predictive power.

[0119] Determine model performance. After the model training is completed, that is, after all the training data has been processed through each round of iteration, turn to the validation set to test the performance of the model. The core goal of this step is to evaluate the prediction accuracy of the model to see whether it meets the preset expectations. If the model exhibits excellent performance and meets the established requirements, it can directly enter the model evaluation stage. However, if the performance of the model does not meet the expected standards, it is necessary to consider improving the model. At this point, the present invention should directly enter the model improvement process and proceed to optimize and adjust the model. On the contrary, if it is evaluated that the model does not need further improvement, it can be selected to adjust the hyperparameters for subsequent model operations.

[0120] Adjusting hyperparameters. In order to further optimize the performance of the model, the present invention needs to carefully adjust the hyperparameters to find the best combination. These hyperparameters cover the learning rate, batch size, number of iterations, regularization coefficient, and key parameters in the neural network architecture, such as the number of layers and nodes. Accurately setting these hyperparameters can not only significantly improve the training efficiency of the model, but also enhance its generalization ability, thereby effectively avoiding the overfitting problem. After the adjustment is completed, the present invention will re-execute the model training process to verify and further optimize the settings of the hyperparameters until the model performance reaches the optimal state.

[0121] Model improvement. In the process of improving the performance of deep learning models, the present invention adopts a series of iterative optimization strategies. These strategies include fine-tuning the network structure, optimizing parameters, selecting activation functions, applying regularization techniques, introducing data enhancement methods, and updating optimization algorithms. By constantly trying different methods and techniques, the present invention seeks best practices in order to improve the generalization ability and robustness of the model. In the step of model construction, the present invention redesigns the network architecture, which may include increasing or decreasing the number of layers, adjusting the number or type of neurons. At the same time, the present invention carefully adjusts the parameters of the model, such as learning rate, batch size, etc., to ensure that the model can effectively learn from the data. Selecting an appropriate activation function is crucial for the nonlinear expression of the model, so this aspect will also be explored. In addition, in order to reduce the overfitting phenomenon, the present invention adopts regularization techniques, such as L1, L2 regularization or Dropout. Data enhancement technology improves the generalization ability of the model by increasing the diversity of training data. In the selection of optimization algorithms, the effects of different algorithms on the model training speed and convergence are also considered. Through these comprehensive optimization strategies, the present invention aims to improve the effect of deep learning models in practical applications and ensure that the model can perform tasks accurately and stably.

[0122] Model evaluation. When evaluating deep learning models, in addition to the key indicator of accuracy, the present invention also needs to pay attention to the time complexity and space complexity of the model. Time complexity measures the efficiency of the computing resources required by the model in the reasoning and training stages, that is, the reasoning speed and the length of training time. It reflects the advantages and disadvantages of the response speed and training efficiency of the model in practical applications. On the other hand, space complexity focuses on the space occupied by the model in memory or storage, specifically the size of the model and the memory usage during runtime. This is crucial for evaluating the feasibility and scalability of the model in a resource-constrained environment. Taking into account the time complexity and space complexity, the present invention can more comprehensively evaluate the efficiency and practicality of the model, and then select the model that best suits the specific application scenario. This comprehensive evaluation method helps to ensure that the model can run efficiently and stably in actual deployment.

[0123] Perform face quality analysis and rating based on output results. Extract trace features such as texture, direction, and length from the model output. Calculate lithology classification indices using methods such as BQ, Q system, or GSI. Rating the face lithology quality based on the classification indices. Generate a report containing analysis results, ratings, and recommendations. Then, geologists review the analysis results to ensure the accuracy of the ratings.

[0124] The above-described embodiment is only a preferred embodiment of the present invention and does not impose any form of limitation on the present invention. Other possible variants and improved forms can also be explored on the basis of following the technical solutions described in the claims. These variants and modifications also fall within the scope of the present invention.

[0125] Summarize

[0126] The present invention provides a tunnel face lithology analysis method that integrates images and three-dimensional point clouds, and aims to improve the accuracy and efficiency of lithology analysis. Traditional lithology analysis methods mainly rely on visual observation by engineers, which is not only inefficient but also easily affected by human factors. In order to overcome these shortcomings, the present invention adopts a method that combines image processing technology and three-dimensional point cloud technology. Specifically, a binocular camera is first used to obtain a high-quality image of the face and generate a three-dimensional point cloud model. Next, the image and point cloud data are preprocessed, and the structural surface traces of the face are extracted using advanced algorithms. On this basis, a deep learning model is trained in combination with expert annotation data to achieve automatic segmentation and recognition of traces. By optimizing and integrating the model, the two-dimensional and three-dimensional joint features of the face are further extracted, and the lithology classification index is calculated. Finally, the method can significantly improve the accuracy and efficiency of lithology analysis, and provide important support for technological progress in the field of tunnel engineering construction. This lithology analysis method that integrates image and three-dimensional point cloud technology not only improves the automation of data processing, but also reduces the interference of human factors, and has broad application prospects.

Claims

1. A tunnel face lithology analysis method by integrating images and three-dimensional point clouds, characterized by the following steps: include: Step 1) Using the two-dimensional tunnel face image collected by the binocular camera, the image data is three-dimensionally reconstructed to obtain three-dimensional point cloud data of the tunnel face; Step 2) preprocessing the two-dimensional tunnel face image data obtained in step 1) to obtain two-dimensional image data of the tunnel face structure trace; Step 3) preprocessing the three-dimensional point cloud data of the tunnel face obtained in step 1); Step 4) extracting features from the three-dimensional point cloud data of the tunnel face preprocessed in step 3) to obtain three-dimensional point cloud data of the tunnel face structure trace; Step 5) fusing and analyzing the two-dimensional image data of the tunnel face structure trace extracted in step 2) and the three-dimensional point cloud data of the tunnel face structure trace, and modifying them through expert human intervention to form a labeling database; Step 6) using the two-dimensional image data of the tunnel face preprocessed in step 2) and the three-dimensional point cloud data of the tunnel face preprocessed in step 3) as inputs of the deep learning model; using the data in the annotation database as labels of the deep learning model to perform model training; Step 7) Output two-dimensional joint data and three-dimensional joint characteristics according to the deep learning model, calculate the lithology classification index, and complete the lithology classification; The input of the deep learning model in step 6) is the two-dimensional image data and the three-dimensional point cloud data, and the output of the deep learning model is the automatic segmentation and recognition of the track line of the tunnel face; based on the output of the deep learning model, the track line of the joint is generated from the point cloud using Hough transform; adjacent track lines are connected to form a complete track line of the joint; In the deep learning model, the image data and the 3D point cloud data are respectively obtained by the image feature extraction module and the point cloud feature extraction module to obtain the features of the image data and the features of the palm face point cloud; then the boundary features are refined and extracted by the boundary attention module to obtain the image features and the point cloud features respectively. FI and voxel features FV ; Next, use the voxel image cross-modal association module VI to convert the image features FI and voxel features FV to integrate; Then, the fused features are input into a multi-layer perceptron to output the final segmentation result, which includes the segmented trace texture; Finally, the segmented results are fused and matched, and the results of the image data are used to enrich the results of the point cloud data.

2. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, characterized in that It also includes step 8) expert human intervention to correct the results output by deep learning, add them to the annotation database, and carry out cyclic learning.

3. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, characterized in that The deep learning model is the BASEG model; the image feature extraction module is ResNet.

4. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, wherein In step 2), the image data obtained in step 1 is preprocessed including: image correction, noise removal, image sharpening, edge detection and image registration; The image correction methods are histogram equalization and gamma correction; The noise removal method is median filtering and / or Gaussian filtering; The methods of image sharpening include Laplacian operator and unsharp masking; Image registration uses feature matching and image refinement to extract structural surface joints.

5. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, wherein In step 3), preprocessing the three-dimensional point cloud data obtained in step 1) includes: First, mean filtering, median filtering, Gaussian smoothing filtering and point cloud resampling are used; normal estimation and scale adjustment are used to enhance the point cloud; Then use the iterative closest point ICP algorithm and normal distribution transform NDT algorithm to perform point cloud registration and alignment.

6. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, wherein In step 4), the steps of extracting features from the tunnel face point cloud data include: First, the triangulated network is established using the Delaunay triangulation algorithm; Then, the normal tensor voting (NTV) method is used to detect feature points and detect the crack feature points of the tunnel face in the point cloud data. Then, a point cloud compression algorithm based on the Laplacian operator is used to shrink the feature points; Next, the feature points are grouped and connected; Finally, the feature points are linearized and the linearized trajectory segments are connected to complete the extraction of the structural surface traces of the tunnel face from the point cloud data.

7. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, wherein In step 5), first, the face structure trace image is registered with the point cloud data through feature point matching and geographic coordinate conversion, that is, the points on the two-dimensional image are matched with the points in the three-dimensional space; Then, experts identify key features on the face based on the fused data and annotate these key features on the 2D image and 3D point cloud data; Finally, the annotated image and point cloud data and related attribute information are stored in the database.

8. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, characterized in that The deep learning model includes the following modules: Image feature extraction module, which extracts features of input image data through 2D CNN; Point cloud feature extraction module, which extracts the features of the input face point cloud by converting the point cloud data into voxels through Voxel Encoder; The edge attention mechanism processes the two-dimensional image module BAI, which processes the features of the image data to obtain image features FI ; The module BAP, which processes the 3D point cloud with the edge attention mechanism, processes the features of the face point cloud to obtain voxel features FV ; Voxel image cross-modal association module VI, image features FI and voxelized point cloud features FV Fusion is performed. In this module: first, the image features corresponding to the voxel center are obtained, and then the image features are sampled using the offset; the voxel features are processed as queries, and the sampled image features are represented as keys and values, which are then sent to the multi-head cross attention module to obtain image-enhanced voxel features; the image-enhanced voxel features are concatenated with the original features to obtain the final fused features; Then, the final fusion features are input into the multi-layer perceptron to output the final segmentation result; Finally, the segmentation results are fused and matched, and the results of the image data are used to enrich the results of the point cloud data. This step is manually intervened by experts to manually compare and correct the final segmentation results to make the segmentation results more accurate.

9. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, characterized in that The training steps of the deep learning model are: First, the traces extracted using traditional methods are used as "Ground truth"; Then, the two-dimensional image data and three-dimensional point cloud data preprocessed in step 2) and step 3) are divided into a training set, a validation set and a test set; Next, conduct model training, evaluation, and optimization; Use the training set to train the model, calculate the output through forward propagation, and then update the weights through the back-propagation algorithm; after each epoch, use the validation set to evaluate the model performance; optimize the model by adjusting the number of convolutional layers, using different activation functions, and adjusting the network depth; The model is evaluated based on its performance on the validation set, and is continuously optimized, with the best weights finally saved for trace extraction.

10. The method for analyzing lithology of a tunnel face by fusing images and three-dimensional point clouds according to claim 1, characterized in that In step 7), for the features of the traces extracted from the output of the deep learning model, the lithology classification index is calculated using the BQ method, the Q system method or the GSI method; According to the classification index, the quality of the lithology of the tunnel face is graded and the analysis results are generated; Geological experts then review the analysis to ensure the accuracy of the rating.

Images