Lithology identification method based on the fusion of optical characteristics and Mohs hardness

Abstract

The present invention discloses a lithology identification method based on the fusion of optical features and Mohs hardness, including constructing an image classification model, loading pre-training weights, simulating Mohs hardness through an MSI method, and adding it to the image classification model; constructing an MFI-TF model, inputting batch samples, outputting a prediction value as a sequence, and taking the index value of the maximum value in the prediction sequence as the prediction category; judging whether the batch samples are samples in the training set, if so, calculating the loss value according to the prediction sequence and the actual category, and updating the weight of the model; otherwise calculating the accuracy of the prediction; judging whether the current accuracy is higher than the optimal accuracy of historical training, if so, saving the optimal model weight; otherwise repeating the training until all models are trained, and selecting the top n models according to the accuracy for constructing a CVCR model, and outputting the final lithology prediction result after the training is completed. The present invention can realize automatic lithology identification and improve the recognition efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of rock and mineral natural image recognition, and more specifically, to a rock property recognition method based on the fusion of optical features and Mohs hardness. Background Art

[0002] Natural rock and mineral image recognition and classification can provide more efficient assistance for field rock and mineral exploration. It is of great significance for the category identification of rock and mineral samples in different environments and for non-professionals to understand rock and mineral knowledge.

[0003] Generally, the identification of the lithology of natural rocks and minerals requires professionals to use relevant means and experiments to make judgments. This method is time-consuming and labor-intensive and requires high literacy of the identification personnel. Due to the introduction of deep learning methods in the field of natural rock and mineral images, the research on automatic identification of lithology has been developed. There are many existing studies on lithology identification, which use camera imaging and deep image classification models to complete the automatic lithology identification process. However, there are still some problems. First, the image of natural lithology, that is, the image of rock and mineral under natural conditions directly captured by optical photography equipment (camera), is affected by many factors, including the inconsistency of imaging brightness, angle, clarity, etc. caused by weather, geographical location and humidity; second, the lithology reference basis specified by traditional lithology identification contains many characteristic information that is not available in natural images, such as the structural characteristics, composition and hardness of rocks. Therefore, the existing automatic identification of lithology is in a bottleneck state and it is difficult to form a usable application. In addition, the identification of natural rocks and minerals relies more on the support of data sets, and requires feature learning based on a large amount of data, while requiring the authenticity and reliability of the data to ensure the reliability of the learning results. This is because the deep learning method uses image feature induction to find similar or shared feature information between samples from a large number of samples, and uses this to complete the recognition work.

[0004] The existing technology is mainly based on the transfer convolutional network model and its improved algorithm for identification, such as: 1) A method for intelligent recognition of rock specimen images, application number 202210504978.6; 2) A method, device, equipment and storage medium for rock and mineral identification, application number 202011270883.X; 3) A rock classification method, terminal equipment and storage medium, application number 202111329893.0; 4) A method and device for rapid classification and identification of rock and mineral lithology based on artificial intelligence, application number 202110256098.7; 5) Automatic rock category identification method based on deep learning and Bayesian network, application number 201910509533.5; 6) A rock and mineral thin section image recognition method based on residual shrinkage module and attention mechanism, application number 202110674575.1. The two mainstream classification processes and frameworks are CNN (Convolutional Neural Network) and ViT (Vision Transformer). The former has long been the main research framework for visual tasks, and the latter is a framework derived from the field of natural language processing. With the deepening of current research, ViT has been able to achieve better results than traditional convolutional networks. Although ViT has performed well in many works, its performance in the experiment is far from that of CNN. In addition, it is currently difficult to achieve more significant improvements in lithology recognition methods that rely entirely on CNN, and it is difficult to achieve accurate and reliable recognition for complex lithology recognition scenarios in practical applications. Summary of the invention

[0005] The present invention aims to solve the technical problem that the CNN-based lithology identification method is difficult to cope with the complex and diverse lithology characteristics in actual scenarios. In order to solve this technical problem and cope with the identification scenario with sample dilemma, the technical solution adopted by the present invention is: to provide a lithology identification method based on the fusion of optical features and Mohs hardness, adopt the CNN framework and the latest ViT framework to combine the design model, and introduce the fusion of Mohs hardness index to achieve rapid and accurate identification of lithology of natural rock and mineral images.

[0006] The present invention provides a lithology identification method based on the fusion of optical features and Mohs hardness, which specifically includes the following steps:

[0007] S1: Collect rock data set samples, split them into training set and validation set after preprocessing, and obtain batch samples through data sampling;

[0008] S2: Build an image classification model, load pre-trained weights, simulate Mohs hardness through the MSI method, and add it to the image classification model;

[0009] S3: Build an MFI-TF model, input batch samples into the MFI-TF model, output the predicted value as a sequence, and use the index value of the maximum value in the predicted sequence as the predicted category;

[0010] S4: Determine whether the batch samples are samples in the training set. If so, the predicted sequence and the actual category are subjected to the cross entropy loss function to obtain the loss value, and the loss value is used to update the model weight; otherwise, it is a sample in the validation set, and the accuracy of the prediction result is calculated;

[0011] S5: Determine whether the current accuracy is higher than the optimal accuracy of historical training. If so, save the optimal model weight; otherwise, repeat steps S3-S5 to complete the training of all models, and rank the models according to the accuracy of the prediction results to select the top n models;

[0012] S6: The n models obtained in step S5 are used to construct a CVCR model, and the CVCR model is trained, and the final lithology prediction result is output through the trained CVCR model.

[0013] Preferably, in step S1, rock data set samples are collected by a camera device.

[0014] Furthermore, step S1 also includes: sorting out the lithology corresponding to the rocks in the rock data set sample, and collecting the Mohs hardness range of the lithology.

[0015] Furthermore, the image classification model constructed in step S2 includes a CNN framework model and a ViT architecture model.

[0016] Optionally, the CNN framework model includes: one or more of VGG, AlexNet, GoogleNet, ResNet, ResNeXt, ShuffleNet and GhostNet.

[0017] Optionally, in step S2, constructing the image classification model includes:

[0018] A ViT architecture model is constructed according to an open source network structure. The ViT architecture model includes one or more of ViT, T2TViT, CVT, CCT and PVT, and a SAM module of the ViT architecture model is optimized using an SSAM method.

[0019] Furthermore, the use of the SSAM method to optimize the SAM module of the ViT architecture model specifically includes:

[0020] The original SAM module contains the operation of formula 1:

[0021] f sa :=Softmax(XQKX)XV Formula 1

[0022] In formula 1, f sa represents the self-attention calculation function, X is the input feature, Q is the query matrix, K is the key-value matrix, V is the value matrix, and Softmax is a function that converts the input vector into probability, as shown in Formula 2:

[0023]

[0024] In formula 2, X is the input feature, i is the index of input feature X, and C is the maximum index of X;

[0025] The SSAM method is specifically based on the original SAM module, and performs the following operations on the Q and K matrices in Formula 1 in advance:

[0026]

[0027] In formula 3, ops represents the operation function, W is the input, is the input mean. Formula 3 is used to normalize all Q and K matrices before calculating the similarity.

[0028] Furthermore, in step S3, the step of constructing the MFI-TF model specifically includes:

[0029] a) Connect the output of the image classification model and the MSI to the Concat layer, where the input of the image classification model is the image sample and the input of the MSI is the Mohs hardness range;

[0030] b) Create several sets of sequentially connected Linear layers, Batch Normalization layers, and Activation layers, and stack them into a multi-layer perceptron MLP, whose input is connected to the Concat layer;

[0031] c) Create a Linear layer with the output dimension set to the number of target categories and its input connected to the MLP.

[0032] Furthermore, in step S6, the step of training the CVCR model specifically includes:

[0033] a) Create several filters to filter and identify the correct samples;

[0034] b) stack all models with one filter between them;

[0035] c) Input the samples from the first model to the last model. There is filtered data between models. The input data of each model includes the filtered data of the previous model and a batch of data for this training. These two parts of data are input into the next model together.

[0036] d) Finally, the prediction results of all models are weighted and accumulated to obtain the final prediction result;

[0037] e) Refer to steps S3-S4 to train and save the model.

[0038] Furthermore, after step S6, the method further includes:

[0039] The lithology prediction results are visualized on the user terminal device.

[0040] The technical solution provided by the present invention has the following beneficial effects:

[0041] The present invention adopts the method introduced by Mohs hardness, which can improve the reliability of rock identification without increasing the difficulty of user use. Considering the CNN architecture model and the ViT architecture model at the same time can reduce the work of the manual screening model, avoid people's subjective selection, and cause the actual identification to fail to achieve the most effective result. In addition, considering that the CNN architecture model focuses on the extraction and recognition of local information, and the sample image actually input is affected by the actual shooting equipment and environmental factors, it may bring a lot of interference to the effective information of the data, such as rock and mineral images obtained by field surveys. In most cases, such images will inevitably shoot non-target rock and mineral information into the image, which is easy to cause CNN recognition to produce preferences and pay too much attention to non-rock and mineral parts. Therefore, the present invention can also use the ViT architecture model, which extracts key information from global information and screens information through the attention mechanism, so as to better pay attention to the characteristics of the target rock and mineral. However, considering that the ViT architecture model will show a situation where it is difficult to converge in rock and mineral identification, the present invention performs SSAM optimization on the attention mechanism in the architecture model. Based on the above technical solution of the present invention, accurate and rapid identification of the lithology of natural rock and mineral images can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0043] Figure 1 It is a technical flow chart of a lithology identification method based on the fusion of optical features and Mohs hardness in an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of the structure of the MFI-TF model in an embodiment of the present invention;

[0045] Figure 3 Schematic diagram of the structure of the CVCR model in the embodiment of the present invention. DETAILED DESCRIPTION

[0046] To make the purpose, technical solution and advantages of the present invention clearer, the following will further describe the embodiments of the present invention in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] Please refer to Figure 1 The present invention provides a lithology identification method based on the fusion of optical features and Mohs hardness, and its main steps are as follows:

[0048] S1: Collect rock data set samples, split them into training set and validation set after preprocessing, and obtain batch samples through data sampling;

[0049] Specifically, use camera equipment (resolution not less than 640×480) to collect image samples of physical rocks. To ensure the reliability of identification, the number of samples of each category should be no less than 500, and sort out the lithology corresponding to the rock, and collect the Mohs hardness range of the lithology with a resolution of not less than 640×480;

[0050] The preprocessing process is as follows: the pre-prepared data set samples (images and Mohs hardness ranges collected by cameras and other devices) are cropped and resized, and then divided into training sets and validation sets in a ratio of 8:2. After sampling operations (usually 32 or 64 samples per batch, which can be adjusted to 128, 256, 512, etc. according to the hardware equipment conditions), batch samples are taken from the data set (training set or validation set);

[0051] S2: Build an image classification model, load pre-trained weights, simulate Mohs hardness through the MSI method, and add it to the image classification model;

[0052] It should be noted that the MSI method in this link is based on the fact that the optical characteristics and hardness characteristics of rocks are independent of each other. By simulating the hardness distribution, the model has the function of hardness perception. Specifically, it generates hardness values ​​corresponding to the image based on the range of Mohs hardness of each category of lithology. The generation method is Gaussian distribution sampling in the interval. The sampling distribution formula in the interval [a, b] is as follows:

[0053]

[0054] Where r is a random number and S is a sample value in the interval [a, b].

[0055] When a range is input into the model, a Mohs hardness simulation value that conforms to the Gaussian distribution can be generated as the hardness characteristic of the sample.

[0056] Specifically, the constructed image classification model includes a CNN framework model and a ViT architecture model;

[0057] For the CNN framework model, step S2 is as follows:

[0058] The CNN framework model was constructed according to the open source network structure, the pre-trained weights were loaded, the Mohs hardness was simulated by the MSI method (Mohs' Hardness Simulation Identification Method), and then added to the CNN framework model.

[0059] In this embodiment, the CNN framework model is preferably a VGG model, and other CNN framework models may also be selected, such as AlexNet, GoogleNet, ResNet, ResNeXt, ShuffleNet, GhostNet, etc.

[0060] For the ViT architecture model, step S2 specifically includes:

[0061] The ViT architecture model is constructed according to the open source network structure. The model can be selected from ViT, T2TViT, CVT, CCT, PVT, etc., and the SSAM (Stabilizing Attention Mechanism Module) method is used to optimize the SAM (Self-Attention Mechanism) module of the ViT architecture model.

[0062] The steps of optimizing the SAM module of the ViT architecture model by using the SSAM method specifically include:

[0063] First locate the SAM module, which implements the operations including formula 1:

[0064] f sa :=Softmax(XQKX)XV Formula 1

[0065] In formula 1, f sa represents the self-attention calculation function, X is the input feature, Q is the query matrix, K is the key-value matrix, and V is the value matrix. These matrices all contain n vectors, and the n vectors correspond to the n samples of the batch input. The three matrices are obtained by processing the input data through the Linear layer. Softmax is a function that converts the input vector into a probability, as shown in Formula 2:

[0066]

[0067] In formula 2, X is the input feature, i is the index of input feature X, and C is the maximum index of X;

[0068] The SSAM method is based on SAM and performs the following operations on the Q and K matrices in advance:

[0069]

[0070] In formula 3, ops represents the operation function, W is the input, is the input mean. Formula 3 is used to normalize all Q and K matrices before calculating the similarity.

[0071] Through the above operations, the problem that the ViT architecture model is difficult to converge in rock and mineral identification can be solved, and the efficiency of identification can be improved.

[0072] S3: Based on the constructed multiple image classification models (including several CNN framework models and ViT architecture models), an MFI-TF (Mohs' Hardness Simulation Identification And Transfer Learning Method) model is constructed, batch samples are input into the MFI-TF model, the output prediction value is a sequence, and the index value of the maximum value in the prediction sequence is used as the predicted category;

[0073] like Figure 2 As shown, in this embodiment, step S3 of constructing the MFI-TF model specifically includes:

[0074] a) Connect the output of the image classification model and the MSI to the concatenation layer, where the input of the image classification model is the image sample and the input of the MSI is the Mohs hardness range;

[0075] b) Create several sets of linear layers, batch normalization layers, and activation layers, and stack them into a multi-layer perceptron (MLP), whose input is connected to the concatenation layer;

[0076] c) Create a linear layer (Linear) with the output dimension set to the number of target categories and its input connected to the multi-layer perceptron (MLP).

[0077] It should be noted that after the image sample passes through the image classification network, a one-dimensional optical feature vector can be obtained, and after the MSI method, a Mohs hardness value can be obtained. After the Mohs hardness value and the optical feature vector are spliced, they are input into the multi-layer perceptron (MLP) to fuse and classify the optical and hardness features. Its structure diagram is shown in Figure 2 As shown in the figure, the image classification network loads the weights pre-trained on the ImageNet dataset for initialization to complete the transfer learning.

[0078] S4: Determine whether the batch samples are samples in the training set. If so, the predicted sequence and the actual category are subjected to the cross entropy loss function to obtain the loss value, and the loss value is used to update the model weight; otherwise, it is a sample in the validation set, and the accuracy of the prediction result is calculated;

[0079] S5: Determine whether the current accuracy is higher than the optimal accuracy of historical training. If so, save the optimal model weight; otherwise, repeat steps S3-S5 to complete the training of all models, and rank the models according to the accuracy of the prediction results to select the top n models;

[0080] S6: The n models obtained in step S5 are used to construct a CVCR (CNN and ViT combined recognition strategy) model, and the CVCR model is trained to output the final lithology prediction result.

[0081] It should be noted that, regarding the above-mentioned image classification model Classification Model (CM), a mode combining CNN and ViT is used for lithology identification. First, multiple models (including several CNN and ViT models) are used as CM, and the recognition training of the data set is completed through the MFI-TF model method, and finally several MFI-TF models are obtained. We screen out the n MFI-TF models with the highest recognition accuracy. All MFI-TF models are assembled through the CVCR method to form an integrated model E. The integration method is shown in the figure below. Among them, each MFI-TF model uses a Filter filter to complete indirect data transmission. The filter removes the samples predicted correctly in the output of the previous model and only retains the samples predicted incorrectly. In this way, the samples predicted incorrectly by the previous model are input into the next model together with the input samples, and the samples that are incorrectly recognized by the previous model are paid more attention to by the subsequent model. The final prediction result is the weighted cumulative sum of the common prediction results of all intermediate models.

[0082] like Figure 3 As shown, in this embodiment, the step of training the CVCR model in step S6 specifically includes:

[0083] a) Create several filters to filter and identify the correct samples;

[0084] b) Stack all models (Model 1-Model n) with one filter between them;

[0085] c) Input the samples (Data) from the first model to the last model. There is also filtered data between models. The input data of each model includes the filtered data of the previous model and a batch of data for this training. These two parts of data are input into the next model together;

[0086] d) Finally, the prediction results of all models (Predict 1-Predict n) are combined through learnable weights (weight values ​​w1-w n is the parameter to be learned, which will be automatically determined during the training process) for weighted accumulation to obtain the final prediction result;

[0087] e) Refer to steps S3-S4 to train and save the model.

[0088] Based on the above-mentioned lithology identification method, an embodiment of the present invention further provides a lithology identification device based on the fusion of optical features and Mohs hardness. The device specifically includes: an MSI module, an MFI-TF module, a CVCR module and an SSAM module. Each module of the device corresponds to the specific steps of implementing the above-mentioned lithology identification method.

[0089] It should be noted that the device is suitable for hardware support of mobile devices and needs to include a usable camera with a resolution of no less than 640×480 and an operating system no less than Android 4.0.

[0090] The device is applied to the lithology recognition of natural rock and mineral images, and the recognition results are visualized, which specifically includes the following steps:

[0091] 1. Select a suitable server and build a trained model on it to provide lithology identification services to the user's mobile phone or device;

[0092] 2. Design a front-end user interaction interface for users to take photos and upload data (the device or the camera on the mobile phone can be used);

[0093] 3. Guide the user to estimate the rock hardness through the setting method. The specific steps are as follows:

[0094] a) Rub the rock with your hand to determine whether the rock can be easily rubbed off. If so, it means that the hardness range is between [0,1]; otherwise, proceed to the next step;

[0095] b) Use your fingernail to scratch the rock to determine whether the rock can be clearly scratched by the fingernail. If it can, the hardness range is (1,2.5]; otherwise, proceed to the next step;

[0096] c) Use a key or other iron object to scratch the rock to determine whether the rock can be clearly scratched by the iron. If so, the hardness range is (2.5,5]; otherwise, proceed to the next step;

[0097] d) Use a ceramic piece to scratch the rock to determine whether the rock can be clearly scratched by the ceramic piece. If it can, the hardness range is (5,7]; otherwise, the hardness range is greater than 7;

[0098] e) In particular, if the rock contains larger particles, such as boulder (particle diameter> 256 mm) conglomerate, large boulder (particle diameter 64-256 mm) conglomerate, pebble (particle diameter 4-64 mm) conglomerate, etc., it means that the Mohs hardness is not suitable and the range should be set to the empty interval;

[0099] 4. Upload user data to the server, use the model to identify lithology, and feed back the results to the user's front end, which will be displayed to the user after page rendering;

[0100] 5. Save the image samples uploaded by the user, user information and the recognition results of the model to the server.

[0101] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0102] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.

[0103] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A rock property identification method based on the fusion of optical features and Mohs hardness, characterized in that: The following steps are involved: S1: Collect rock data set samples, split them into training set and validation set after preprocessing, and obtain batch samples through data sampling; S2: Build an image classification model, load pre-trained weights, simulate Mohs hardness through the MSI method, and add it to the image classification model; S3: Build an MFI-TF model, input batch samples into the MFI-TF model, output the predicted value as a sequence, and use the index value of the maximum value in the predicted sequence as the predicted category; S4: Determine whether the batch samples are samples in the training set. If so, the predicted sequence and the actual category are subjected to the cross entropy loss function to obtain the loss value, and the loss value is used to update the model weight; otherwise, it is a sample in the validation set, and the accuracy of the prediction result is calculated; S5: Determine whether the current accuracy is higher than the optimal accuracy of historical training. If so, save the optimal model weight; otherwise, repeat steps S3-S5 to complete the training of all models, and rank the models according to the accuracy of the prediction results to select the top n models; S6: using the n models obtained in step S5 to construct a CVCR model, and training the CVCR model, and outputting the final lithology prediction result through the trained CVCR model; Image classification models include CNN framework model and ViT architecture model; The steps of constructing the MFI-TF model specifically include: a) Connect the output of the image classification model and the MSI to the Concat layer, where the input of the image classification model is the image sample and the input of the MSI is the Mohs hardness range; b) Create several sets of sequentially connected Linear layers, Batch Normalization layers, and Activation layers, and stack them into a multi-layer perceptron MLP, whose input is connected to the Concat layer; c) Create a Linear layer with the output dimension set to the number of target categories and its input connected to the MLP; The steps of training the CVCR model specifically include: a) Create several filters to filter and identify the correct samples; b) stack all models with one filter between them; c) Input the samples from the first model to the last model. There is filtered data between models. The input data of each model includes the filtered data of the previous model and a batch of data for this training. These two parts of data are input into the next model together. d) Finally, the prediction results of all models are weighted and accumulated to obtain the final prediction result; e) Refer to steps S3-S4 to train and save the model.

2. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 1 is characterized in that: In step S1, rock data set samples are collected through a camera device.

3. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 1 is characterized in that: Step S1 also includes: sorting out the lithology corresponding to the rocks in the rock data set sample, and collecting the Mohs hardness range of the lithology.

4. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 1 is characterized in that: The CNN framework model includes: one or more of VGG, AlexNet, GoogleNet, ResNet, ResNeXt, ShuffleNet and GhostNet.

5. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 1 is characterized in that: In step S2, the image classification model is constructed, including: A ViT architecture model is constructed according to an open source network structure, wherein the ViT architecture model includes one or more of ViT, T2T ViT, CVT, CCT and PVT, and a SAM module of the ViT architecture model is optimized using an SSAM method.

6. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 5 is characterized in that: The SAM module of the ViT architecture model is optimized by using the SSAM method, specifically including: The original SAM module contains the operation of formula 1: f sa :=Softmax(XQKX)XV Formula 1 In formula 1, f sa represents the self-attention calculation function, X is the input feature, Q is the query matrix, K is the key-value matrix, V is the value matrix, and Softmax is a function that converts the input vector into probability, as shown in Formula 2: In formula 2, X is the input feature, i is the index of input feature X, and C is the maximum index of X; The SSAM method is specifically based on the original SAM module, and performs the following operations on the Q and K matrices in Formula 1 in advance: In formula 3, ops represents the operation function, W is the input, is the input mean. Formula 3 is used to normalize all Q and K matrices before calculating the similarity.

7. The lithology identification method based on the fusion of optical features and Mohs hardness according to claim 1 is characterized in that: After step S6, the method further includes: The lithology prediction results are visualized on the user terminal device.

Images