Method, device and equipment for identifying sepiolite in rock and medium

Abstract

The invention relates to a method, device and equipment for identifying sepiolite in rock and a medium, and belongs to the technical field of image processing, and the method for identifying sepiolite in rock is used for identifying sepiolite in rock based on a fully trained hybrid neural network model. The hybrid neural network model comprises a feature extraction layer, an early fusion layer, a middle fusion layer and a late fusion layer, feature extraction is carried out on the multi-modal data based on the feature extraction layer, image feature vectors and text feature vectors are spliced based on the early fusion layer, weighted fusion is carried out on the multi-modal feature vectors based on the middle fusion layer, and a fusion result is obtained; and performing classification feature extraction on the fused feature vector based on the late fusion layer, fusing the image classification feature and the text classification feature, and inputting the fused classification feature, the fused feature vector and the multi-modal feature vector into a classifier to identify the sepiolite in the rock. And the identification efficiency and the identification precision of the sepiolite in the rock are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device, equipment and medium for identifying sepiolite in rocks. Background Art

[0002] Sepiolite has a unique physicochemical structure. Due to its high-temperature resistance and strong adsorption properties, it is widely used in environmental protection, new energy, and new materials. In recent years, sepiolite has also attracted widespread attention in the geological field. In the oil and gas sector, sepiolite's excellent adsorption properties can effectively absorb large amounts of organic matter, contributing to the formation of abundant unconventional oil and gas reservoirs. Furthermore, it can promote the dolomitization of high-lying limestone, transforming it into high-quality dolomite reservoirs, giving rise to two novel oil and gas accumulation models: self-generation and self-reservoir, and lower generation and upper reservoir. By studying the thickness and distribution of sepiolite layers, it is possible to infer the fluctuations of paleogeology and predict favorable distribution areas for oil and gas reservoirs, providing important geological evidence for oil and gas exploration. Furthermore, the formation and distribution of sepiolite can also reflect various paleoenvironmental information, such as evaporation systems, water level changes, climatic conditions, and anoxic environments, revealing significant geological events. Therefore, the rapid and accurate identification of sepiolite in rocks is considered an effective tool for oil and gas reservoir exploration, paleogeomorphological reconstruction, and paleoenvironmental restoration.

[0003] Traditional identification of sepiolite mainly relies on manual microscopic observation and chemical analysis. However, due to the variable color and occurrence of sepiolite, it has problems such as low efficiency and low accuracy, making it difficult to adapt to the needs of large-scale and high-precision detection. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, device, equipment and medium for identifying sepiolite in rocks to solve the technical problems of low efficiency and low accuracy in identifying sepiolite.

[0005] In order to solve the above problems, in a first aspect, the present invention provides a method for identifying sepiolite in rocks, which is used to identify sepiolite in rocks based on a well-trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; The method for identifying sepiolite in the rock comprises: Acquiring multimodal data to be identified, the multimodal data comprising image data and spectral data rich in sepiolite; Performing feature extraction on the multimodal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; splicing the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; Performing weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; Based on the late fusion layer, classification features are extracted from the fused feature vector to obtain image classification features and text classification features. The image classification features and text classification features are fused, and the fused classification features, the fused feature vector and the multimodal feature vector are input into a classifier to identify sepiolite in the rock.

[0006] In one possible implementation, the hybrid neural network model includes a ResNet model, a 3D-CNN network, and an LSTM network; and the feature extraction layer is used to extract features from the multimodal data to obtain an image feature vector and a text feature vector, including: Preprocessing the multimodal data to obtain a rock sample data set; Performing image feature extraction on the rock sample dataset based on the ResNet model to obtain an image feature vector; A convolution operation is performed on the rock sample data set based on the 3D-CNN network to obtain a feature map that integrates spatiotemporal information, and feature extraction is performed on the feature map that integrates spatiotemporal information based on the LSTM network to obtain a text feature vector.

[0007] In a possible implementation, preprocessing the multimodal data to obtain a rock sample dataset includes: Performing denoising processing on the image data by using Gaussian filtering, performing image enhancement on the denoised image data by using a histogram equalization method, and performing normalization processing on the enhanced image data to complete preprocessing of the image data; Using a peak detection algorithm to identify characteristic peaks of the spectral data, and calculating the integrated area of ​​the characteristic peaks to obtain a characteristic peak set related to sepiolite; Using the characteristic peak set as a reference anchor point, dynamic time warping is used to perform sequence alignment on the spectral data, and the spectral data after sequence alignment is truncated to complete the preprocessing of the spectral data; The preprocessed image data and spectral data are merged, and the merged data are processed using multivariate scatter correction and standard normal variation to obtain a rock sample data set.

[0008] In a possible implementation, the concatenating the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector includes: Performing local feature extraction on the image feature vector through a multi-layer convolution module of a ResNet model to obtain image features; Extracting features from the text feature vector using an LSTM network to obtain text features; The image features and text features are concatenated to obtain a multimodal feature vector.

[0009] In a possible implementation, performing weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector includes: Performing convolution and pooling operations on the image feature vector of the multimodal feature vector through the convolution module and pooling module of the ResNet model, respectively, to obtain image features; Extracting features from the text feature vector of the multimodal feature vector using the LSTM network to obtain text features; The image features and the text features are weightedly fused to obtain a fused feature vector.

[0010] In a possible implementation, the late fusion layer is used to extract classification features from the fused feature vector to obtain image classification features and text classification features, the image classification features and the text classification features are fused, and the fused classification features, the fused feature vector, and the multimodal feature vector are input into a classifier to identify sepiolite in the rock, including: Performing classification feature extraction on the fused feature vector based on the ResNet model to obtain image classification features; Performing classification feature extraction on the fused feature vector based on the LSTM network to obtain text classification features; The image classification features and text classification features are fused using a weighted average method. The fused feature vector and the multimodal feature vector are aligned with the fused classification features through a fully connected layer and then input into a Softmax classifier to identify sepiolite in rocks.

[0011] In one possible implementation, the loss function of the hybrid neural network model is: , , in, is the cross entropy loss function, is the mean square error loss function, is the probability distribution of the true label, is the probability distribution predicted by the model, For samples or events, is the sample size, For the The true value of the sample, For the The predicted value of the sample.

[0012] In a second aspect, the present invention further provides a device for identifying sepiolite in rocks, which is used to identify sepiolite in rocks based on a fully trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; the device for identifying sepiolite in rocks includes: a data acquisition module, configured to acquire multimodal data to be identified, wherein the multimodal data includes image data and spectral data rich in sepiolite; A feature extraction module, configured to perform feature extraction on the multimodal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; An early fusion module, configured to concatenate the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; a mid-term fusion module, configured to perform weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; A late fusion module is used to extract classification features of the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, fuse the image classification features and text classification features, and input the fused classification features, the fused feature vector and the multimodal feature vector into a classifier to identify sepiolite in the rock.

[0013] In a third aspect, the present invention further provides an electronic device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps of the method for identifying sepiolite in rocks as described above are implemented.

[0014] In a fourth aspect, the present invention also provides a computer-readable storage medium for storing computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the method for identifying sepiolite in rocks described in any one of the above-mentioned method items.

[0015] The beneficial effects of the present invention are: obtaining multimodal data to be identified, the multimodal data including image data and spectral data rich in sepiolite, providing a data basis for the training of a hybrid neural network model, performing feature extraction and feature fusion on the multimodal data through the feature extraction layer, early fusion layer, mid-term fusion layer and late fusion layer of the hybrid neural network model, gradually transitioning from low-level features to high-level features, enabling the hybrid neural network model to better capture subtle differences between different samples, significantly improving the generalization ability of the hybrid neural network model, and improving the accuracy and efficiency of sepiolite identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For the technical personnel of the present invention, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A flow chart of an embodiment of a method for identifying sepiolite in rocks provided by the present invention; Figure 2 A schematic diagram of sepiolite under a microscope for the method of identifying sepiolite in rocks provided by the present invention; Figure 3 A schematic diagram of the XRD pattern characteristics of sepiolite in the method for identifying sepiolite in rocks provided by the present invention; Figure 4 A schematic diagram of the infrared spectrum of sepiolite in the method for identifying sepiolite in rocks provided by the present invention; Figure 5 A schematic diagram showing a comparison of the Adam optimizer for the method for identifying sepiolite in rocks provided by the present invention; Figure 6 A schematic diagram of the ROC curve of the method for identifying sepiolite in rocks provided by the present invention; Figure 7 A schematic structural diagram of an embodiment of a device for identifying sepiolite in rocks provided by the present invention; Figure 8 This is a schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0019] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0020] Before presenting the embodiments, the following terms are explained.

[0021] Dynamic Time Warping (DTW) is an algorithm used to measure the similarity between two time series. Its main feature is that it can align local deformations on the time axis, thereby addressing the problems of scaling or speed differences in the time dimension of the series.

[0022] Peak detection algorithm: It is an algorithm that locates the peak (local maximum point) by identifying the maximum value of the signal in a local neighborhood. Its core idea is to use a sliding window to traverse the signal, compare the amplitude of the data points within each window range, and mark the local maximum that meets the conditions as a candidate peak.

[0023] The present invention discloses a method, device, equipment, and medium for identifying sepiolite in rocks, which can be used in a computer. The method, equipment, or computer-readable storage medium involved in the present invention can be integrated with the above-mentioned equipment or can be relatively independent.

[0024] A specific embodiment of the present invention discloses a method for identifying sepiolite in rocks, which can be executed by a computer, specifically by one or more processors of the computer. Figure 1 As shown, the method for identifying sepiolite in rocks is used to identify sepiolite in rocks based on a well-trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; the method for identifying sepiolite in rocks includes: S101, acquiring multimodal data to be identified, the multimodal data including image data and spectral data rich in sepiolite; It should be noted that comprehensive information collection on rock samples rich in sepiolite was conducted to obtain multimodal data to be identified, revealing the morphological characteristics of the rock samples as well as the chemical composition and mineral spectrum characteristics of sepiolite. Multimodal data provides a basis for model training.

[0025] S102, performing feature extraction on the multimodal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; It should be noted that the hybrid neural network model includes the ResNet model, the 3D-CNN network, and the LSTM network. The feature extraction layer extracts image feature vectors and text feature vectors. S103, concatenating the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; S104, performing weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; S105. Extracting classification features from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, fusing the image classification features and the text classification features, and inputting the fused classification features, the fused feature vector, and the multimodal feature vector into a classifier to identify sepiolite in the rock. It should be noted that the image feature vectors and text feature vectors are processed through the feature extraction layer, early fusion layer, mid-term fusion layer and late fusion layer of the hybrid neural network model, gradually transitioning from low-level features to high-level features. The early fusion layer, mid-term fusion layer and late fusion layer fully utilize the pre-training advantages of ResNet on large-scale image datasets. Through the transfer learning strategy, the general features are transferred to the sepiolite recognition task, which reduces the dependence on limited sepiolite sample data and effectively alleviates the overfitting problem. At the same time, the unique ability of LSTM in processing sequence data enables the model to better capture the subtle differences between different samples, thereby significantly improving the generalization ability of the model. It not only improves the accuracy of sepiolite identification, but also enhances the robustness and reliability of the model in practical applications. In the hybrid neural network model, ResNet focuses on extracting key visual features such as the spatial morphology and texture of sepiolite from rock images, thereby simplifying the complexity of LSTM in processing sequence data. At the same time, LSTM provides ResNet with the ability to analyze sequence data, helping the hybrid neural network model to more accurately identify the characteristic patterns of sepiolite in images. The synergistic effect of LSTM and ResNet enables the model to deeply analyze the sequence characteristic variation patterns of sepiolite crystal structure and chemical composition, and extract deeper sequence features, thereby achieving higher accuracy and efficiency in the sepiolite identification process.

[0026] In some embodiments, in step S101, multimodal data to be identified is obtained, and the multimodal data includes image data and spectral data rich in sepiolite. Rock samples rich in sepiolite are obtained. Through field investigations and extensive collection, rock samples rich in sepiolite from different regions and different periods are sorted out. The rock samples cover a variety of lithologies, including argillaceous limestone, limestone, sepiolite mudstone, sepiolite shale, and sepiolite claystone. The rock samples containing sepiolite in different regions are shown in Table 1. Table 1

[0027] The rock samples were analyzed in multiple dimensions. During the analysis, infrared spectroscopy was used to detect the vibration behavior of water molecules and hydroxyl groups inside the sepiolite. Its characteristic absorption peak usually appears at 3500-3700 The XRD spectrum of sepiolite has a series of unique diffraction peaks, and the distribution and intensity of these peaks are closely related to its crystal structure. The combined analysis of infrared spectroscopy and XRD can effectively distinguish sepiolite from clay minerals such as montmorillonite and kaolinite. Scanning electron microscopy (SEM) can clearly show the fibrous or layered micromorphology of sepiolite, while energy dispersive spectroscopy (EDS) can accurately determine the content of elements such as magnesium, silicon, and oxygen in sepiolite. Combining SEM and EDS analysis, we can fully reveal the morphological characteristics and elemental composition of sepiolite, thereby obtaining multimodal data to be identified. The multimodal data includes image data rich in sepiolite and spectral data. For a schematic diagram of sepiolite under the microscope, please refer to Figure 2 ,like Figure 2 As shown, a is pure sepiolite observed under natural light conditions, b is sepiolite under cross-polarized light, c is continuous sepiolite fibers under transmission electron microscopy, d is a transmission electron microscopy image showing the orderly arrangement of asbestos-shaped sepiolite, e is sepiolite aggregates, f is fibrous sepiolite, g is a mat formed by longer sepiolite fibers, and h is a thick aggregate of sepiolite; for a schematic diagram of the XRD spectrum characteristics of sepiolite, please refer to Figure 3 , the infrared spectrum diagram of its sepiolite, please refer to Figure 4 ,like Figure 4 As shown, a is the infrared spectrum of pure sepiolite, b is the infrared spectrum of sepiolite after adsorbing methyl green, c is the infrared spectrum of sepiolite after adsorbing crystal violet, and d is the infrared spectrum of sepiolite after adsorbing methylene blue.

[0028] In some embodiments, in step S102, feature extraction is performed on the multimodal data based on the feature extraction layer to obtain image feature vectors and text feature vectors, and its hybrid neural network model includes a ResNet model, a 3D-CNN network, and an LSTM network; first, the image data and spectral data in the multimodal data are preprocessed to obtain a rock image data set, and targeted preprocessing methods are used for different types of data to improve data quality. For the collected image data, Gaussian filtering is used to denoise the image data to eliminate noise interference in the image, and the denoised image data is enhanced by a histogram equalization method, that is, the contrast and clarity of the image are enhanced, and the enhanced image data is normalized, that is, the image data is normalized to interval to complete the preprocessing of image data; for spectral data, the peak detection algorithm (local maximum method) is used to identify the characteristic peaks of the spectral data, and the integral area of ​​the characteristic peaks is calculated to obtain the characteristic peak set related to sepiolite; that is, the characteristic peaks related to sepiolite are screened out, and irrelevant peaks are eliminated. The characteristic peak set is used as the reference anchor point, and the spectral data is sequenced by dynamic time warping to ensure that the characteristic peak positions of all spectral data are consistent. The spectral data after sequence alignment is truncated to keep the spectral length consistent, thus completing the preprocessing of spectral data; the preprocessed image data and spectral data are merged, and the spectral data are analyzed. The merged data were processed using multivariate scattering correction and standard normal variation to obtain a rock sample dataset, which includes image data, spectral data, and geological attributes. After preprocessing the image and spectral data, the image and spectral data were merged, and multivariate scattering correction technology was further used to correct for spectral differences caused by scattering effects and highlight the characteristic absorption peaks of sepiolite. Standard normal variable transformation was used to eliminate the effects of factors such as sample concentration and particle size on spectral intensity. At the same time, smoothing filtering and wavelet transform were used to reduce noise, improve the signal-to-noise ratio of the spectrum, and further optimize data quality.

[0029] Secondly, the image feature extraction of the rock sample dataset is performed based on the ResNet model to obtain the image feature vector. In the hybrid neural network model, ResNet is used as the core architecture for image feature extraction. The image data in the rock sample dataset is input into the ResNet model. After layer-by-layer processing of multi-layer convolution, pooling and residual modules, the convolution layer uses convolution kernels of various sizes to slide on the image to extract local details such as edges and textures, and gradually transition from low-level features to high-level features. Then, the pooling layer downsamples the convolution result to reduce the computational burden and simplify the model. Subsequently, the residual module adds the input and convolution output through jump connections, effectively alleviating the deep network. To address the vanishing and exploding gradient problems, global average pooling compresses the feature map into a fixed-length vector, outputs the classification result through a fully connected layer, and obtains the image feature vector. The model can extract high-level features of the image in the deep structure. High-level features can accurately capture the semantic information in the image, including key elements such as the shape, color, and texture of the object. In particular, the unique porous honeycomb structure of sepiolite provides an important basis for distinguishing sepiolite from other rock types. ResNet, through multi-layer convolution and pooling structures, can effectively capture the spatial characteristics of sepiolite. At the same time, its unique shortcut connection mechanism avoids the vanishing gradient problem and ensures the stability and efficiency of deep network training.Finally, a convolution operation is performed on the rock sample dataset based on the 3D-CNN network to obtain a feature map that integrates spatiotemporal information. Feature extraction is performed on the feature map that integrates spatiotemporal information based on the LSTM network to obtain a text feature vector. For the IR, XRD, SEM and EDS serialized data in the spectral data, the IR, XRD, SEM and EDS serialized data are input into the LSTM network and combined with the 3D convolutional neural network (3D-CNN) to capture the dynamic characteristics of sepiolite changing over time. LSTM, with its gating mechanism of input gate, forget gate and output gate, can effectively regulate information transmission, accurately capture long-distance dependencies in text, and generate a fixed-dimensional text feature vector. Specifically, the bidirectional encoding model (BERT) based on Transformer generates context-related word vectors, and the word vectors are input into the 3D convolutional neural network (3D- To capture the dynamic characteristics of sepiolite over time, a CNN (Convolutional Neural Network) and LSTM network are used to capture the dynamic characteristics of sepiolite. 3D convolution treats text as a sequence consisting of both time and feature dimensions. By sliding the convolution kernel across multiple channels, convolution operations are performed on neighboring words and their features, generating a feature map that integrates spatiotemporal information. After receiving the feature map, the LSTM uses memory cells and a gating mechanism to process the information. Memory cells store long-term and short-term information, while the gating mechanism regulates the flow of information. The LSTM reads the feature map by time step, filters key information through gating, and deeply explores the semantic and contextual relationships of the text, integrating these relationships into the word vector. Finally, the fully connected layer outputs the text feature vector, which is processed by the LSTM. Thanks to its gating structure, the LSTM effectively captures long-term dependencies and extracts key features from the map, helping the model gain a deep understanding of the crystal structure and composition of sepiolite.

[0030] In some embodiments, in step S103, the image feature vector and the text feature vector are spliced ​​based on the early fusion layer to obtain a multimodal feature vector. The ResNet model includes a multi-layer convolution module, a pooling module and a residual module. The early fusion layer splices the image feature vector and the text feature vector to obtain a multimodal feature vector, specifically: the image feature vector is extracted locally by the multi-layer convolution module of the ResNet model to obtain image features, the text feature vector is extracted by the LSTM network to obtain text features, and the image features and the text features are spliced ​​to obtain a multimodal feature vector; the early fusion layer is in the hybrid neural network model. In the shallow fusion of this type, at the data input stage, the pixel matrix and numerical features are merged according to rules by splicing the feature vectors of the image, spectrum and geological background. The fused input data is then sent to a unified network structure containing ResNet and LSTM, that is, the early fusion layer. The convolution layer of the ResNet model in the early fusion layer is responsible for extracting the local features of the image part, while the LSTM network uses its sequence processing capability to capture time series or contextual information. During the training process, the entire network is optimized in an end-to-end manner, prompting ResNet and LSTM to work together to learn feature representations that are helpful in identifying sepiolite.

[0031] In some embodiments, in step S104, the multimodal feature vector is weightedly fused based on the mid-term fusion layer to obtain a fused feature vector, and the multimodal feature vector is input into the mid-term fusion layer. The mid-term fusion layer fuses the image feature vector and the text feature vector in the multimodal feature vector through weighted fusion, specifically: the image feature vector of the multimodal feature vector is convolved and pooled respectively through the convolution module and the pooling module of the ResNet model to obtain image features, and the text feature vector of the multimodal feature vector is feature extracted through the LSTM network to obtain text features; the image features and the text features are fused through weighted fusion to obtain a fused feature vector, and the mid-term fusion is performed. The fusion layer is the middle layer of the hybrid neural network model. First, the rock data is sent to independent ResNet and LSTM networks for feature extraction. ResNet uses multi-layer convolution and pooling operations to extract intermediate features such as edges and textures from rock images; LSTM processes sequence data and mines time-dependent or context-related features. Subsequently, in the middle layer of the network, the features extracted by ResNet and LSTM are combined through weighted fusion, and weights are assigned to different features according to task requirements. The fused features are input into the subsequent late fusion layer network layer for further learning and transformation of features, and finally the recognition results of sepiolite are output through the classifier. During training, the entire network participates in optimization and adjusts parameters to improve recognition performance.

[0032] In some embodiments, in step S105, classification features are extracted from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, the image classification features and text classification features are fused, and the fused classification features, the fused feature vector and the multimodal feature vector are input into the classifier to identify sepiolite in the rock. In the final fusion stage of the hybrid neural network model, the late fusion layer extracts classification features from the fused feature vector based on the ResNet model to obtain image classification features; extracts classification features from the fused feature vector based on the LSTM network to obtain text classification features; and fuses the image classification features and the text classification features using a weighted average method, and performs feature comparison between the fused feature vector and the multimodal feature vector and the fused classification features through the fully connected layer. After alignment, it is input into the Softmax classifier to identify sepiolite in the rock, and the fused feature vector is input into the late fusion layer. ResNet and LSTM independently complete the forward propagation calculation of the rock data. ResNet processes the image data through convolution, pooling and residual modules to generate feature representations for classification; LSTM processes the sequence data, uses memory units to store long-term and short-term information, and regulates the flow of information through the gating mechanism, and finally outputs the classification features. Subsequently, the output features of ResNet and LSTM are fused by the weighted average method, and the multimodal feature vector of the early fusion layer and the fused feature vector of the mid-fusion layer are aligned with the fused classification features of the late fusion layer through the fully connected layer, and then input into the Softmax classifier to determine whether sepiolite exists in the rock.

[0033] A hybrid neural network model was trained based on a rock image dataset. Feature extraction was performed on the rock image dataset to obtain image feature vectors and text feature vectors. These feature vectors were then fused using the early, mid, and late fusion layers of the hybrid neural network model to identify sepiolite in rocks. After preprocessing the spectral and image data, core features that significantly contribute to sepiolite identification were screened, including spatial, spectral, geometric, and temporal features. A multi-level deep learning framework of early, mid, and late fusion in the hybrid neural network model was used to centrally process multi-source data and complement the advantages of these features, significantly improving the model's recognition and discrimination capabilities. The training process of the hybrid neural network model involved dividing the rock sample dataset into a training set, a validation set, and a test set, with ratios of 70%, 15%, and 15%, respectively. The training and validation sets were then fed into the hybrid neural network model. The hybrid neural network model was trained based on the training set, and its hyperparameters were adjusted using the validation set to obtain a fully trained hybrid neural network model. The performance of the fully trained hybrid neural network model was evaluated using the test set.

[0034] In the feature fusion stage of the hybrid neural network model (early fusion layer, mid-term fusion layer, and late fusion layer), the image features extracted by ResNet are combined with the sequence features processed by LSTM. A feature splicing and fusion strategy is adopted to enable the model to simultaneously utilize the dual information of image and sequence data. This dual information accurately identifies the characteristic patterns related to sepiolite in the image, thereby improving the accuracy and efficiency of sepiolite identification.

[0035] During the model training phase, we selected corresponding loss functions for different tasks. For the binary classification task of whether sepiolite exists, we used the cross entropy loss function; for the regression task of predicting sepiolite content, we used the mean square error loss function. Using the Adam high-efficiency optimizer, we conducted multiple rounds of iterative training based on the training data. We also used the validation set to monitor model performance in real time and adjust hyperparameters in a timely manner to prevent overfitting. For a comparison diagram of the Adam optimizer, please refer to Figure 5 ,like Figure 5 As shown in the figure, the comparison chart of different learning rates and weight decay parameters of the Adam optimizer. The horizontal coordinate is the number of training times of the hybrid neural network model, and the vertical coordinate is the loss value of the hybrid neural network model. When the loss value is lower than 0.1%, the training is stopped ( represents the learning rate, represents the weight decay parameter), and on the test set, the test set is predicted based on the fully trained hybrid neural network model to obtain prediction results, which are evaluated by accuracy, recall rate, and mean square error. Through a comprehensive evaluation of the model, focusing on its performance in sepiolite identification, content prediction, and crystal structure analysis, the model is continuously optimized to achieve the best state, thereby ensuring the accurate identification of sepiolite in rocks. The loss function is: , , in, is the cross entropy loss function, is the mean square error loss function, is the probability distribution of the true label, is the probability distribution predicted by the model, For samples or events, is the sample size, For the The true value of the sample, For the The predicted value of the sample.

[0036] In order to comprehensively evaluate the performance of the model, a variety of indicators are used, including recall rate, F1 score and area under the ROC curve (AUC). For a schematic diagram of the ROC curve, please refer to Figure 6 By using cross-validation technology, the impact of data partitioning on the evaluation results is effectively reduced. For key indicators of the model training process, please refer to Table 2. Table 2

[0037] The model parameters were carefully adjusted based on the evaluation indicators, and the overall strategy was optimized, which significantly improved the model's performance in identifying sepiolite in rocks. In addition, according to the needs of actual application scenarios, structural parameters such as the number of ResNet layers, convolution kernel size, and the number of hidden layers and neurons in LSTM were flexibly adjusted to adapt to data sets of different sizes and recognition tasks of different complexity.

[0038] Based on the well-trained hybrid neural network model, the rock samples to be predicted are predicted to identify sepiolite in the rock. The recognition accuracy of high-purity sepiolite in the rock samples reached 98%, medium-purity was 92%, and low-purity was 90%.

[0039] In summary, the method for identifying sepiolite in rocks provided by the present invention is used to identify sepiolite in rocks based on a well-trained hybrid neural network model, the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer and a late fusion layer, based on the feature extraction layer, feature extraction is performed on multimodal data to obtain image feature vectors and text feature vectors, based on the early fusion layer, the image feature vectors and the text feature vectors are spliced ​​to obtain a multimodal feature vector, based on the mid-term fusion layer, weighted fusion is performed on the multimodal feature vectors to obtain a fused feature vector, based on the late fusion layer, classification feature extraction is performed on the fused feature vector to obtain image classification features and text classification features, the image classification features and the text classification features are fused, and the fused classification features, the fused feature vector and the multimodal feature vector are input into a classifier to identify sepiolite in rocks, thereby improving the recognition efficiency and recognition accuracy of sepiolite in rocks.

[0040] In order to better implement the method for identifying sepiolite in rocks in the embodiment of the present invention, based on the method for identifying sepiolite in rocks, correspondingly, Figure 7 As shown, an embodiment of the present invention further provides a device for identifying sepiolite in rocks, which is used to identify sepiolite in rocks based on a fully trained hybrid neural network model. The hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer. The device 700 for identifying sepiolite in rocks includes: A data acquisition module 701 is used to acquire multimodal data to be identified, where the multimodal data includes image data and spectral data rich in sepiolite; A feature extraction module 702 is used to extract features from multimodal data based on a feature extraction layer to obtain image feature vectors and text feature vectors; An early fusion module 703 is used to concatenate the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; A mid-term fusion module 704 is configured to perform weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; The late fusion module 705 is used to extract classification features from the fused feature vector based on the late fusion layer, obtain image classification features and text classification features, fuse the image classification features and text classification features, and input the fused classification features, fused feature vector and multimodal feature vector into the classifier to identify sepiolite in the rock.

[0041] like Figure 8As shown, the present invention also provides an electronic device 800 , which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palmtop computer, or a server. The electronic device 800 includes a processor 801 , a memory 802 , and a display 803 . Figure 8 Only some of the components of the electronic device 800 are shown, but it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0042] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard drive or memory within electronic device 800. In other embodiments, memory 802 may also be an external storage device within electronic device 800, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, memory 802 may include both an internal storage unit of electronic device 800 and an external storage device. Memory 802 is used to store application software installed in electronic device 800 and various data, such as program code installed in electronic device 800. Memory 802 may also be used to temporarily store data that has been output or is about to be output. In one embodiment, memory 802 stores a program for identifying sepiolite in rocks, which can be executed by processor 801, thereby implementing the methods for identifying sepiolite in rocks according to various embodiments of the present invention.

[0043] In some embodiments, the processor 801 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 802 , such as a method for identifying sepiolite in rocks.

[0044] In some embodiments, display 803 can be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display identification information from the rock sepiolite identification program and to display a visual user interface. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0045] In some embodiments, when the processor 801 executes the program for identifying sepiolite in rocks in the memory 802, the various steps of the method for identifying sepiolite in rocks as described in the above embodiments are implemented. Since the method for identifying sepiolite in rocks has been described in detail above, it will not be repeated here.

[0046] Accordingly, the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps or functions of the method for identifying sepiolite in rocks provided in the above-mentioned method embodiments can be implemented.

[0047] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0048] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or replacements that can be easily conceived by any technician familiar with the technical neighbors within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for identifying sepiolite in rocks, characterized in that: for identifying sepiolite in rocks based on a well-trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; The method for identifying sepiolite in the rock comprises: Acquiring multimodal data to be identified, the multimodal data comprising image data and spectral data rich in sepiolite; Performing feature extraction on the multimodal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; splicing the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; Performing weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; Based on the late fusion layer, classification features are extracted from the fused feature vector to obtain image classification features and text classification features. The image classification features and text classification features are fused, and the fused classification features, the fused feature vector and the multimodal feature vector are input into a classifier to identify sepiolite in the rock.

2. The method for identifying sepiolite in rocks according to claim 1, characterized in that: The hybrid neural network model includes a ResNet model, a 3D-CNN network, and an LSTM network; and the feature extraction layer is used to extract features from the multimodal data to obtain image feature vectors and text feature vectors, including: Preprocessing the multimodal data to obtain a rock sample data set; Performing image feature extraction on the rock sample dataset based on the ResNet model to obtain an image feature vector; A convolution operation is performed on the rock sample data set based on the 3D-CNN network to obtain a feature map that integrates spatiotemporal information, and feature extraction is performed on the feature map that integrates spatiotemporal information based on the LSTM network to obtain a text feature vector.

3. The method for identifying sepiolite in rocks according to claim 2, characterized in that: Preprocessing the multimodal data to obtain a rock sample data set includes: Performing denoising processing on the image data by using Gaussian filtering, performing image enhancement on the denoised image data by using a histogram equalization method, and performing normalization processing on the enhanced image data to complete preprocessing of the image data; Using a peak detection algorithm to identify characteristic peaks of the spectral data, and calculating the integrated area of ​​the characteristic peaks to obtain a characteristic peak set related to sepiolite; Using the characteristic peak set as a reference anchor point, dynamic time warping is used to perform sequence alignment on the spectral data, and the spectral data after sequence alignment is truncated to complete the preprocessing of the spectral data; The preprocessed image data and spectral data are merged, and the merged data are processed using multivariate scatter correction and standard normal variation to obtain a rock sample data set.

4. The method for identifying sepiolite in rocks according to claim 2, characterized in that: The ResNet model includes a multi-layer convolution module, a pooling module, and a residual module; the image feature vector and the text feature vector are concatenated based on the early fusion layer to obtain a multimodal feature vector, including: Performing local feature extraction on the image feature vector through a multi-layer convolution module of a ResNet model to obtain image features; Extracting features from the text feature vector using an LSTM network to obtain text features; The image features and text features are concatenated to obtain a multimodal feature vector.

5. The method for identifying sepiolite in rocks according to claim 4, characterized in that: The weighted fusion of the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector includes: Performing convolution and pooling operations on the image feature vector of the multimodal feature vector through the convolution module and pooling module of the ResNet model, respectively, to obtain image features; Extracting features from the text feature vector of the multimodal feature vector using the LSTM network to obtain text features; The image features and the text features are weightedly fused to obtain a fused feature vector.

6. The method for identifying sepiolite in rocks according to claim 5, characterized in that: The method further comprises: extracting classification features from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, fusing the image classification features and the text classification features, and inputting the fused classification features, the fused feature vector, and the multimodal feature vector into a classifier to identify sepiolite in the rock, including: Performing classification feature extraction on the fused feature vector based on the ResNet model to obtain image classification features; Performing classification feature extraction on the fused feature vector based on the LSTM network to obtain text classification features; The image classification features and text classification features are fused using a weighted average method. The fused feature vector and the multimodal feature vector are aligned with the fused classification features through a fully connected layer and then input into a Softmax classifier to identify sepiolite in rocks.

7. The method for identifying sepiolite in rocks according to claim 6, characterized in that: The loss function of the hybrid neural network model is: ， ， in, is the cross entropy loss function, is the mean square error loss function, is the probability distribution of the true label, is the probability distribution predicted by the model, For samples or events, is the sample size, For the The true value of the sample, For the The predicted value of the sample.

8. A device for identifying sepiolite in rocks, characterized in that: for identifying sepiolite in rocks based on a well-trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; The device for identifying sepiolite in rocks comprises: a data acquisition module, configured to acquire multimodal data to be identified, wherein the multimodal data includes image data and spectral data rich in sepiolite; A feature extraction module, configured to perform feature extraction on the multimodal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; An early fusion module, configured to concatenate the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector; a mid-term fusion module, configured to perform weighted fusion on the multimodal feature vectors based on the mid-term fusion layer to obtain a fused feature vector; A late fusion module is used to extract classification features of the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, fuse the image classification features and text classification features, and input the fused classification features, the fused feature vector and the multimodal feature vector into a classifier to identify sepiolite in the rock.

9. An electronic device, characterized in that: including memory and processor; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps of the method for identifying sepiolite in rocks according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps of the method for identifying sepiolite in rocks as described in any one of claims 1 to 7.

Images