A multi-modal lithology identification method and system
Patent Information
- Application Number
- CN202411093095.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2026-08-21
- Estimated Expiration
- 2044-08-09
AI Technical Summary
近年来国内外学者借助图像识别技术对岩性识别开展了一系列研究，但尚未建立较为完善的隧道内岩性智能识别方法，缺乏针对工程应用的便携式岩性识别设备，同时对围岩岩性识别的准确度和速度都有待提高
[0030](1)本发明建立基于岩石图像的深度学习模型对围岩岩性进行识别，能够对岩石特征进行自动提取，并对应岩性关系形成映射，从而达到自动识别的目的，该方法识别准确度高、速度快、成本低，降低了对施工人员专业知识和鉴别经验的要求。
Smart Images
Figure CN119152360B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of geotechnical and tunnel engineering technology, specifically to a multimodal lithology identification method and system. Background Technology
[0002] Tunnel construction frequently encounters unfavorable geological formations such as faults, karst caves, and underground rivers, which can trigger large-scale geological disasters like water inrushes and mudslides. Faults and karst formations are the biggest culprits and challenges in tunnel disasters. Geological forecasting allows for the timely detection of anomalies, providing a basis for adjusting construction plans, effectively preventing tunnel accidents, and ensuring construction safety.
[0003] Lithology identification is a fundamental and crucial issue in geology, resource exploration, tunnel construction, rock mechanics, and engineering, directly impacting the selection of construction schemes and the reliability of reservoir evaluation. Traditional lithology identification methods include (1) visual observation, (2) thin section identification, and (3) physical testing, all of which rely on professional experience and are time-consuming. With the advancement of computer technology, artificial intelligence has begun to be applied to lithology identification, and intelligent lithology identification methods based on data mining and image recognition are constantly emerging. In recent years, scholars at home and abroad have conducted a series of studies on lithology identification using image recognition technology, but a relatively complete intelligent lithology identification method for tunnels has not yet been established, and there is a lack of portable lithology identification equipment for engineering applications. At the same time, the accuracy and speed of surrounding rock lithology identification need to be improved. Summary of the Invention
[0004] To address the problems existing in the prior art, this invention provides a multimodal lithology identification method and system, which can perform real-time identification based on collected rock samples and mesoscopic rock images. It enables efficient storage, sharing and collaboration of field data through a remote data sharing platform, which helps to improve the speed and accuracy of field lithology identification.
[0005] The technical solution of the present invention is as follows:
[0006] In a first aspect of the present invention, a multimodal lithology identification method is provided, comprising the following steps:
[0007] Acquire rock sample images and mesoscopic images of rocks;
[0008] The acquired images are preprocessed, and the preprocessed rock sample images and rock mesoscopic images are input into the trained fusion model for real-time recognition.
[0009] The fusion model outputs lithology identification results, and the images are named according to the identification results;
[0010] The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion.
[0011] In some embodiments of the present invention, the attention-based convolutional neural network introduces an attention mechanism into the convolutional layer, enabling the network to pay more attention to color and texture information in the image when extracting features, and at the same time strengthens the feature maps representing color and texture by increasing weights.
[0012] In some embodiments of the present invention, the weighted feature concatenation method assigns weights to the extracted color and texture features and then concatenates all features to obtain a unified feature representation that emphasizes color and texture features. The specific formula is as follows:
[0013] x concat = [x1; x2; ...; x n ]
[0014] The horizontal bracket [;] indicates a join operation, connecting x1, x2, ..., x3 through a horizontal join. n They are concatenated together to form a new feature vector x. concat .
[0015] In some embodiments of the present invention, the training process of the fusion model includes: training the fusion model using data from the training set, updating the model parameters using the backpropagation algorithm and the gradient descent optimization algorithm, evaluating the performance of the trained model using data from the test set, fine-tuning the model based on the evaluation results, and finally applying the optimized convolutional neural network model to an actual lithology identification task to achieve automatic identification of rock types.
[0016] In some embodiments of the present invention, the image preprocessing process includes adjusting the image's brightness, contrast, and color balance.
[0017] In some embodiments of the present invention, the rock sample image is acquired by a camera, and the mesoscopic image of the rock is acquired by an electronic magnifying glass.
[0018] In a second aspect of the invention, a multimodal lithology identification system is provided, comprising:
[0019] The image acquisition module is configured to acquire rock sample images and rock mesoscopic images;
[0020] The lithology identification module is configured to: preprocess the acquired images, and input the preprocessed rock sample images and rock mesoscopic images into the trained fusion model for real-time identification.
[0021] The data interaction module is configured to: output lithology identification results from the fusion model and name the images based on the identification results;
[0022] The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion.
[0023] In some embodiments of the present invention, the image acquisition module, the lithology identification module, and the data interaction module are mounted on a handheld box-type device. The image acquisition module includes a camera and an electronic magnifying glass. The lithology identification module uses a Nano core board and a carrier board. The data interaction module uses a display screen and an HDMI driver board.
[0024] In some embodiments of the present invention, the Nano core board and carrier board are used to provide computing power, and the lithology identification function is realized by deploying the trained fusion model to the Nano development board. The specific deployment steps are as follows:
[0025] Environment configuration: Complete the basic environment configuration for the Nano development board;
[0026] Model conversion converts the original PyTorch-trained model into a TensorRT model suitable for the Nano development board.
[0027] Model invocation uses Python or C++ API to call TensorRT models and implement model inference.
[0028] In some embodiments of the present invention, a WIFI module is also included, which connects the handheld box-type device to a remote data sharing platform.
[0029] One or more technical solutions of the present invention have the following beneficial effects:
[0030] (1) The present invention establishes a deep learning model based on rock images to identify the lithology of surrounding rocks. It can automatically extract rock features and form a mapping with corresponding lithological relationships, thereby achieving the purpose of automatic identification. The method has high accuracy, fast speed and low cost, and reduces the requirements for the professional knowledge and identification experience of construction personnel.
[0031] (2) The present invention can integrate the features such as color, texture and shape of rock sample images with the features such as crystal structure, pore distribution and grain size of rock mesoscopic images through feature fusion, thereby providing more comprehensive and integrated geological information, which helps to analyze the composition and structure of rocks in depth, improve the robustness of the model, and make the model more robust and reliable when facing diverse data.
[0032] (3) This invention integrates the multimodal lithology identification system into a portable handheld device, which does not affect TBM construction during operation. It can reduce the impact of adverse working environments such as darkness, dampness and dust in the tunnel on identification, and provide a lightweight, flexible, accurate and efficient lithology identification technology for on-site construction, realizing automated identification.
[0033] (4) This invention utilizes a remote data sharing platform to realize real-time storage, sharing and collaboration of on-site images, so that data resources can be fully utilized, reducing repetitive labor and corresponding costs such as image acquisition and data collection, and improving the application scope of the system. Attached Figure Description
[0034] Figure 1 This is a flowchart of the multimodal lithology identification method of the present invention;
[0035] Figure 2 This is a structural diagram of the multimodal lithology identification system of the present invention;
[0036] Figure 3(a) is a top view of the handheld box-type device, Figure 3(b) is a perspective view of the handheld box-type device, and Figure 3(c) is a front view of the handheld box-type device.
[0037] Figure 4 This is a schematic diagram of the metal casing of the handheld box-type device of the present invention;
[0038] Figure 5 This is a module diagram of the handheld box-type device of the present invention.
[0039] In the diagram: 1. Metal casing; 2. Camera; 3. Nano core board and carrier board; 4. Display screen and HDMI driver board; 5. Battery and charging / discharging board; 6. WIFI module; 7. A shell; 8. B shell; 9. Sealing ring; 10. Screw; 11. Rib; 12. USB interface. Detailed Implementation
[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0041] Example 1
[0042] In a typical embodiment of the present invention, a multimodal lithology identification method is proposed, such as... Figure 1 As shown, it includes the following steps:
[0043] Acquire rock sample images and mesoscopic images of rocks;
[0044] The acquired images are preprocessed, and the preprocessed rock sample images and rock mesoscopic images are input into the trained fusion model for real-time recognition.
[0045] The fusion model outputs lithology identification results, and the images are named according to the identification results;
[0046] The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion.
[0047] Furthermore, the attention-based convolutional neural network introduces an attention mechanism into the convolutional layers, enabling the network to pay more attention to color and texture information in the image when extracting features, while also strengthening the feature maps representing color and texture by increasing weights.
[0048] Furthermore, the weighted feature concatenation method assigns weights to the extracted color and texture features, and then concatenates all features to obtain a unified feature representation that emphasizes color and texture features. The specific formula is as follows:
[0049] x concat = [x1; x2; ...; x n ]
[0050] The horizontal bracket [;] indicates a join operation, connecting x1, x2, ..., x3 through a horizontal join. n They are concatenated together to form a new feature vector x. concat .
[0051] Furthermore, the training process of the fusion model includes: training the fusion model using data from the training set, updating the model parameters using the backpropagation algorithm and gradient descent optimization algorithm, evaluating the performance of the trained model using data from the test set, fine-tuning the model based on the evaluation results, and finally applying the optimized convolutional neural network model to the actual lithology identification task to achieve automatic identification of rock types.
[0052] In this embodiment, the image preprocessing process is as follows: the lithology identification module automatically processes the image using traditional image processing techniques. By increasing or decreasing the overall brightness of the image, the rock features in the image are made more clearly visible. By increasing the local contrast of the image or adjusting the image curves, the contrast between the rock and the background in the image is enhanced to highlight the details and texture features of the rock. By removing color casts or adjusting the color distribution in the image, the hue, saturation, and color temperature of the image are adjusted to make the color of the rock more realistic and accurately reflect the actual situation. The standard for preprocessing is to ensure that the processed image has sufficient clarity and contrast to highlight the structure, texture, and color information of the rock, so that the lithology identification module can accurately extract and analyze rock features, while reducing the influence of noise and background interference.
[0053] In this embodiment, the rock sample image is acquired using a camera, and the mesoscopic image of the rock is acquired using an electronic magnifying glass.
[0054] Example 2
[0055] In a typical embodiment of the present invention, a multimodal lithology identification system is proposed, such as... Figure 2 As shown, it includes:
[0056] The image acquisition module is configured to acquire rock sample images and rock mesoscopic images;
[0057] The lithology identification module is configured to: preprocess the acquired images, and input the preprocessed rock sample images and rock mesoscopic images into the trained fusion model for real-time identification.
[0058] The data interaction module is configured to: output lithology identification results from the fusion model and name the images based on the identification results;
[0059] The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion.
[0060] like Figures 3(a)-3(c) as well as Figure 4 As shown, the image acquisition module, lithology identification module, and data interaction module are mounted on a handheld box-type device. The image acquisition module includes a camera 2 and an electronic magnifying glass. The camera acquires images of the rock sample, and the electronic magnifying glass acquires mesoscopic images of the rock. These images are then transmitted to the lithology identification module for storage. The lithology identification module uses a Nano core board and a carrier board 3 to provide computing power. The lithology identification function is achieved by deploying the trained fusion model onto the Nano development board. The data interaction module uses a display screen and an HDMI driver board 4 to display the identification results and named images. It can store and upload images and identification results, and also includes functions for manually modifying identification results, deleting images, and adding notes, enabling human-computer interaction. A core control module is also included to control the operating status and command transmission of other modules.
[0061] Specifically, the handheld box-type device has a metal outer shell, divided into shell A (7) and shell B (8). Shells A and B are sealed together by a sealing ring (9) and connected by screws (10). The sealing ring (9) is placed in a groove in shell A, and the sealing ring (9) is deformed by positive pressure on the sealing ring (11) of the rear shell. Shell B (8) is designed with a double-wall structure, with the sealing ring (11) being the inner wall. Its parting surface is flat, which facilitates sealing. A USB interface (12) is designed on the outside of shell B (8) for connecting external devices such as electronic magnifiers. The USB interface is designed with a soft rubber plug; its contour is deformed by compression to achieve a sealing effect.
[0062] In this embodiment, the handheld box-type device's display screen and casing are made of high-strength materials, and the device is waterproof, dustproof, and drop-proof through sealing and anti-permeability technologies.
[0063] Furthermore, camera 2 is used to capture images of rock samples. Camera 2 is designed as an automatic zoom camera equipped with a flash, and is designed to be waterproof, dustproof, and shockproof to ensure normal shooting in the dim environment of the tunnel. The camera 2 automatically takes pictures when a "take picture" command is sent through the core control module.
[0064] Furthermore, the Nano core board and carrier board 3 are combined to form the Nano development board, which provides computing power. The lithology identification function is realized by deploying the trained fusion model onto the Nano development board. The specific deployment steps are as follows:
[0065] Environment configuration: Complete the basic environment configuration for the Nano development board;
[0066] Model conversion converts the original PyTorch-trained model into a TensorRT model suitable for the Nano development board.
[0067] Model invocation uses Python or C++ API to call TensorRT models and implement model inference.
[0068] The lithology identification module has several pre-trained fusion models built in, including: built-in models, cloud models, and local models. Built-in models are installed with the software and do not support separate upgrades; cloud models are the latest models released on the server and support separate upgrades; local models are models imported through the model import function.
[0069] Furthermore, a display screen and HDMI driver board 4 are used to display the camera footage and recognition results in real time, and to realize human-computer interaction. A battery and charging / discharging board 5 are used to ensure that the portable rock identification device can work continuously in the tunnel.
[0070] Furthermore, it also includes a WIFI module 6, which connects the handheld box-type device to a remote data sharing platform.
[0071] The module layout of Nano core board and carrier board 2, camera 3, display and HDMI driver board 4, battery and charging / discharging board 5, and WIFI 6 is as follows: Figure 5 As shown.
[0072] The data interaction module adopts a remote data sharing platform, which integrates image and recognition result uploading, storage and Internet-based remote transmission technology, thereby enabling multiple users to share and process images and recognition results.
[0073] The method of using the multimodal portable handheld box-type device provided in this embodiment is as follows:
[0074] A. The core control module sends instructions to the controller to power on the camera 3 and capture images of the rock sample, and to power on the electronic magnifier connected to the USB interface 12 and capture images of the rock surface. The images are then transmitted to the lithology identification module.
[0075] B. Preprocess the acquired images to optimize their quality by adjusting brightness, contrast, and color balance. Traditional image processing techniques are used: increasing or decreasing overall brightness makes rock features more clearly visible; increasing local contrast or adjusting curves enhances the contrast between rocks and background, highlighting rock details and textures; removing color casts or adjusting color distribution adjusts hue, saturation, and color temperature, making the rock colors more realistic and accurate, thus optimizing image quality.
[0076] C. Import the trained cloud model and local model or load the built-in model for lithology identification, and feed the identification results back to the data interaction module.
[0077] D. After recognition is complete, name the image according to the recognition result. If the recognition is inaccurate, you can manually modify the recognition result.
[0078] E. Store and upload the image and recognition results; click Delete to delete the image and recognition results.
[0079] F. Repeat step AE until the lithology of the surrounding rock is identified.
[0080] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.
Claims
1. A multimodal lithology identification method, characterized in that, Includes the following steps: Acquire rock sample images and mesoscopic images of rocks; The acquired images are preprocessed, and the preprocessed rock sample images and rock mesoscopic images are input into the trained fusion model for real-time recognition. The fusion model outputs lithology identification results, and the images are named according to the identification results; The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion. The attention-based convolutional neural network introduces an attention mechanism into the convolutional layers, enabling the network to pay more attention to color and texture information in the image when extracting features, and at the same time strengthens the feature maps representing color and texture by increasing weights.
2. The multimodal lithology identification method as described in claim 1, characterized in that, The weighted feature concatenation method assigns weights to the extracted color and texture features, and then concatenates all features to obtain a unified feature representation that emphasizes color and texture features. The specific formula is as follows: [;] indicates a join operation, which connects horizontally to... , …, They are concatenated together to form a new feature vector. .
3. The multimodal lithology identification method as described in claim 1, characterized in that, The training process of the fusion model includes: training the fusion model with data from the training set, updating the model parameters using the backpropagation algorithm and gradient descent optimization algorithm, evaluating the performance of the trained model with data from the test set, fine-tuning the model based on the evaluation results, and finally applying the optimized convolutional neural network model to a practical lithology identification task to achieve automatic identification of rock types.
4. The multimodal lithology identification method as described in claim 1, characterized in that, The image preprocessing process involves adjusting the image's brightness, contrast, and color balance.
5. The multimodal lithology identification method as described in claim 1, characterized in that, The rock sample images were acquired using a camera, and the mesoscopic images of the rocks were acquired using an electronic magnifying glass.
6. A multimodal lithology identification system employing the multimodal lithology identification method as described in any one of claims 1-5, characterized in that, include: The image acquisition module is configured to acquire rock sample images and rock mesoscopic images; The lithology identification module is configured to: preprocess the acquired images, and input the preprocessed rock sample images and rock mesoscopic images into the trained fusion model for real-time identification. The data interaction module is configured to: output lithology identification results from the fusion model and name the images based on the identification results; The process of establishing the fusion model is as follows: using an image classification algorithm with ShuffleNet as the framework, a convolutional neural network based on an attention mechanism is used to automatically extract color, texture and shape features for rock sample images, and a convolutional neural network is used to automatically extract crystal structure, pore distribution and particle size features for rock mesoscopic images. The two features are fused using a weighted feature concatenation method, and then a SoftMax activation function is used for multi-class classification to obtain a CNN model that integrates feature extraction and fusion.
7. The multimodal lithology identification system as described in claim 6, characterized in that, The image acquisition module, lithology identification module, and data interaction module are installed on a handheld box-type device. The image acquisition module includes a camera and an electronic magnifying glass. The lithology identification module uses a Nano core board and a carrier board. The data interaction module uses a display screen and an HDMI driver board.
8. The multimodal lithology identification system as described in claim 7, characterized in that, The Nano core board and carrier board are used to provide computing power. The lithology identification function is realized by deploying the trained fusion model to the Nano development board. The specific deployment steps are as follows: Environment configuration: Complete the basic environment configuration for the Nano development board; Model conversion converts the original PyTorch-trained model into a TensorRT model suitable for the Nano development board. Model invocation uses Python or C++ API to call TensorRT models and implement model inference.
9. The multimodal lithology identification system as described in claim 7, characterized in that, It also includes a WIFI module that connects the handheld box-type device to a remote data sharing platform.