A method, device, equipment and medium for identifying sepiolite in rock

Abstract

The present application relates to a kind of identification method, device, equipment and medium of sea sponges in rock, belong to image processing technical field, the identification method of sea sponges in rock is used to identify sea sponges in rock based on the complete mixed neural network model of training, mixed neural network model includes feature extraction layer, early fusion layer, mid-term fusion layer and late fusion layer, based on feature extraction layer, feature extraction is carried out to multimodal data, based on early fusion layer, image feature vector and text feature vector are spliced, based on mid-term fusion layer, multimodal feature vector is weightedly fused, based on late fusion layer, classification feature extraction is carried out to the feature vector after fusion, image classification feature and text classification feature are fused, the classification feature after fusion, the feature vector after fusion and multimodal feature vector are input into classifier, to identify out sea sponges in rock, improve the identification efficiency and identification precision of sea sponges in rock.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, device, and medium for identifying sepiolite in rocks. Background Technology

[0002] Sepiolite possesses a unique physicochemical structure. Due to its high-temperature resistance and strong adsorption properties, it is widely used in environmental protection, new energy, and new materials. In recent years, sepiolite has also attracted widespread attention in the geological field. In the oil and gas sector, sepiolite, with its excellent adsorption properties, can effectively adsorb large amounts of organic matter, promoting the rich formation of unconventional oil and gas reservoirs. In addition, it can promote the dolomitization of high-position limestone, thereby transforming it into high-quality dolomite reservoirs, giving rise to two novel oil and gas accumulation models: self-generated and self-retaining, and lower-generated and upper-retaining. By studying the thickness and distribution of sepiolite strata, the undulation characteristics of paleogeography can be inferred, thereby predicting favorable distribution areas of oil and gas reservoirs and providing important geological basis for oil and gas exploration. At the same time, the formation and distribution of sepiolite can also reflect various paleoenvironmental information such as evaporation systems, water level changes, climatic conditions, and anoxic environments, revealing major geological events. Therefore, rapid and accurate identification of sepiolite in rocks is considered an effective means in the fields of oil and gas reservoir exploration, paleogeographic reconstruction, and paleoenvironment restoration.

[0003] Traditional methods for identifying sepiolite mainly rely on manual microscopic observation and chemical analysis. However, due to the variability in color and occurrence of sepiolite, these methods suffer from low efficiency and accuracy, making it difficult to meet the needs of large-scale and high-precision testing. Summary of the Invention

[0004] In view of this, it is necessary to provide a method, apparatus, equipment and medium for identifying sepiolite in rocks, so as to solve the technical problems of low efficiency and low accuracy in identifying sepiolite.

[0005] To address the aforementioned problems, in a first aspect, the present invention provides a method for identifying sepiolite in rocks, which is used to identify sepiolite in rocks based on a fully trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer.

[0006] The method for identifying sepiolite in the rock includes:

[0007] Acquire multimodal data to be identified, the multimodal data including sepiolite-rich image data and spectral data;

[0008] Based on the feature extraction layer, feature extraction is performed on the multimodal data to obtain image feature vectors and text feature vectors;

[0009] The image feature vector and text feature vector are concatenated based on the early fusion layer to obtain a multimodal feature vector;

[0010] The multimodal feature vectors are weighted and fused based on the intermediate fusion layer to obtain the fused feature vectors;

[0011] Based on the late fusion layer, classification features are extracted from the fused feature vector to obtain image classification features and text classification features. The image classification features and text classification features are fused, and the fused classification features, fused feature vector, and multimodal feature vector are input into the classifier to identify sepiolite in rocks.

[0012] In one possible implementation, the hybrid neural network model includes a ResNet model, a 3D-CNN network, and an LSTM network; the step of extracting features from the multimodal data based on the feature extraction layer to obtain image feature vectors and text feature vectors includes:

[0013] The multimodal data is preprocessed to obtain a rock sample dataset;

[0014] Image feature vectors are obtained by extracting image features from the rock sample dataset based on the ResNet model.

[0015] The rock sample dataset is convolutionally processed using the 3D-CNN network to obtain a feature map that integrates spatiotemporal information. Then, the LSTM network is used to extract features from the feature map that integrates spatiotemporal information to obtain a text feature vector.

[0016] In one possible implementation, the preprocessing of the multimodal data to obtain a rock sample dataset includes:

[0017] The image data is denoised using Gaussian filtering, enhanced using histogram equalization, and then normalized to complete the image data preprocessing.

[0018] A peak detection algorithm is used to identify the characteristic peaks of the spectral data and calculate the integral area of ​​the characteristic peaks to obtain a set of characteristic peaks related to sepiolite.

[0019] Using the set of characteristic peaks as the reference anchor point, dynamic time warping is used to align the spectral data sequence, and the aligned spectral data is truncated to complete the preprocessing of the spectral data.

[0020] The preprocessed image data and spectral data were merged, and multivariate scattering correction and standard normal transformation were applied to the merged data to obtain a rock sample dataset.

[0021] In one possible implementation, the step of concatenating the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector includes:

[0022] The image features are obtained by extracting local features from the image feature vector using the multi-layer convolutional module of the ResNet model.

[0023] The text features are obtained by extracting features from the text feature vector using an LSTM network;

[0024] The image features and text features are concatenated to obtain a multimodal feature vector.

[0025] In one possible implementation, the weighted fusion of the multimodal feature vectors based on the intermediate fusion layer to obtain the fused feature vector includes:

[0026] The image features are obtained by performing convolution and pooling operations on the image feature vector of the multimodal feature vector using the convolution and pooling modules of the ResNet model, respectively.

[0027] The text feature vector of the multimodal feature vector is extracted using the LSTM network to obtain text features;

[0028] The image features and the text features are weighted and fused to obtain a fused feature vector.

[0029] In one possible implementation, the late fusion layer is used to extract classification features from the fused feature vector to obtain image classification features and text classification features. These image classification features and text classification features are then fused. The fused classification features, the fused feature vector, and the multimodal feature vector are input into a classifier to identify sepiolite in rocks, including:

[0030] Based on the ResNet model, classification features are extracted from the fused feature vector to obtain image classification features;

[0031] Based on the LSTM network, classification features are extracted from the fused feature vector to obtain text classification features;

[0032] The image classification features and text classification features are fused using a weighted average method. The fused feature vector and multimodal feature vector are then aligned with the fused classification features through a fully connected layer before being input into a Softmax classifier to identify sepiolite in rocks.

[0033] In one possible implementation, the loss function of the hybrid neural network model is:

[0034] ,

[0035] ,

[0036] in, Let cross-entropy be the loss function. Let the mean squared error loss function be . This represents the probability distribution of the true labels. The probability distribution predicted by the model. For samples or events, For the sample size, For the first The true value of each sample For the first The predicted value for each sample.

[0037] Secondly, the present invention also provides a device for identifying sepiolite in rocks, used to identify sepiolite in rocks based on a fully trained hybrid neural network model, wherein the hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer; the device for identifying sepiolite in rocks includes:

[0038] The data acquisition module is used to acquire multimodal data to be identified, including sepiolite-rich image data and spectral data;

[0039] The feature extraction module is used to extract features from the multimodal data based on the feature extraction layer to obtain image feature vectors and text feature vectors;

[0040] An early fusion module is used to concatenate the image feature vector and the text feature vector based on the early fusion layer to obtain a multimodal feature vector;

[0041] The intermediate fusion module is used to perform weighted fusion of the multimodal feature vectors based on the intermediate fusion layer to obtain the fused feature vectors;

[0042] The late fusion module is used to extract classification features from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features. The image classification features and text classification features are fused together, and the fused classification features, fused feature vector, and multimodal feature vector are input into the classifier to identify sepiolite in rocks.

[0043] Thirdly, the present invention also provides an electronic device, comprising: a processor and a memory;

[0044] The memory stores a computer-readable program that can be executed by the processor;

[0045] When the processor executes the computer-readable program, it implements the steps in the method for identifying sepiolite in rocks as described above.

[0046] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps of the method for identifying sepiolite in rocks as described in any of the above-mentioned method items.

[0047] The beneficial effects of this invention are: acquiring multimodal data to be identified, including image data and spectral data rich in sepiolite, providing a data foundation for training a hybrid neural network model; and performing feature extraction and feature fusion on the multimodal data through the feature extraction layer, early fusion layer, mid-term fusion layer, and late fusion layer of the hybrid neural network model, gradually transitioning from low-level features to high-level features, enabling the hybrid neural network model to better capture subtle differences between different samples, significantly improving the generalization ability of the hybrid neural network model, and improving the accuracy and efficiency of sepiolite identification. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an embodiment of the method for identifying sepiolite in rocks provided by the present invention;

[0050] Figure 2 A microscopic illustration of sepiolite in the method for identifying sepiolite in rocks provided by the present invention;

[0051] Figure 3 A schematic diagram of the XRD pattern characteristics of sepiolite in the method for identifying sepiolite in rocks provided by the present invention;

[0052] Figure 4 A schematic diagram of the infrared spectrum of sepiolite in the method for identifying sepiolite in rocks provided by the present invention;

[0053] Figure 5 A comparative schematic diagram of the Adam optimizer for the method of identifying sepiolite in rocks provided by the present invention;

[0054] Figure 6 A schematic diagram of the ROC curve for the method of identifying sepiolite in rocks provided by the present invention;

[0055] Figure 7 A schematic diagram of an embodiment of the sepiolite identification device in rocks provided by the present invention;

[0056] Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0057] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0058] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0059] Before demonstrating the embodiments, the following terms will be explained.

[0060] Dynamic Time Warping (DTW) is an algorithm used to measure the similarity between two time series. Its main feature is that it can align local deformations on the time axis, thereby handling the problem of scaling or velocity differences in the time dimension of the series.

[0061] Peak detection algorithm: This is an algorithm that locates peaks (local maxima) by identifying the maximum values ​​of a signal within a local neighborhood. Its core idea is to use a sliding window to traverse the signal, comparing the amplitude of data points within each window, and marking the local maximum values ​​that meet certain conditions as candidate peaks.

[0062] This invention discloses a method, apparatus, device, and medium for identifying sepiolite in rocks, which can be used in a computer. The method, apparatus, or computer-readable storage medium involved in this invention can be integrated with the aforementioned apparatus or can be relatively independent.

[0063] One specific embodiment of the present invention discloses a method for identifying sepiolite in rocks, which can be executed by a computer, specifically by one or more processors of the computer. For example... Figure 1 As shown, the method for identifying sepiolite in rocks is used to identify sepiolite in rocks based on a fully trained hybrid neural network model. The hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-stage fusion layer, and a late fusion layer. The method for identifying sepiolite in rocks includes:

[0064] S101. Obtain the multimodal data to be identified, including sepiolite-rich image data and spectral data;

[0065] It should be noted that comprehensive information collection was conducted on the sepiolite-rich rock samples to obtain multimodal data for identification. This revealed the morphological characteristics of the rock samples, as well as the chemical composition and mineralogical features of the sepiolite. The multimodal data provided a foundation for model training.

[0066] S102. Based on the feature extraction layer, perform feature extraction on multimodal data to obtain image feature vectors and text feature vectors;

[0067] It should be noted that the hybrid neural network model includes the ResNet model, 3D-CNN network, and LSTM network, which extract image feature vectors and text feature vectors through feature extraction layers;

[0068] S103. Based on the early fusion layer, the image feature vector and the text feature vector are concatenated to obtain a multimodal feature vector;

[0069] S104. Based on the intermediate fusion layer, the multimodal feature vectors are weighted and fused to obtain the fused feature vectors;

[0070] S105. Based on the late fusion layer, the fused feature vector is used to extract classification features to obtain image classification features and text classification features. The image classification features and text classification features are fused. The fused classification features, the fused feature vector, and the multimodal feature vector are input into the classifier to identify sepiolite in the rock.

[0071] It should be noted that the hybrid neural network model processes image and text feature vectors through feature extraction, early fusion, mid-stage fusion, and late fusion layers, gradually transitioning from low-level to high-level features. The early, mid-stage, and late fusion layers fully leverage ResNet's pre-training advantage on large-scale image datasets. Through transfer learning strategies, general features are transferred to the sepiolite recognition task, reducing dependence on limited sepiolite sample data and effectively mitigating overfitting. Simultaneously, LSTM's unique ability to process sequence data allows the model to better capture subtle differences between different samples, significantly improving its generalization ability. Not only does it improve the accuracy of sepiolite identification, it also enhances the robustness and reliability of the model in practical applications. In the hybrid neural network model, ResNet focuses on extracting key visual features such as the spatial morphology and texture of sepiolite from rock images, thereby simplifying the complexity of LSTM processing sequence data. At the same time, LSTM provides ResNet with the ability to analyze sequence data, helping the hybrid neural network model to more accurately identify the feature patterns of sepiolite in images. The synergistic effect of LSTM and ResNet enables the model to deeply analyze the sequence feature variation law of sepiolite crystal structure and chemical composition, extracting deeper sequence features, thereby achieving higher accuracy and efficiency in the process of sepiolite identification.

[0072] In some embodiments, in step S101, multimodal data to be identified is acquired. This multimodal data includes image data and spectral data rich in sepiolite. Sepiolite-rich rock samples are obtained. Through field investigations and extensive collection, sepiolite-rich rock samples from different regions and periods are compiled. These rock samples cover various lithologies, including argillaceous limestone, limestone, sepiolite mudstone, sepiolite shale, and sepiolite claystone. See Table 1 for sepiolite-bearing rock samples from different regions.

[0073] Table 1

[0074]

[0075] Multidimensional analysis was performed on the rock samples. During the analysis, infrared spectroscopy was used to detect the vibrational behavior of water molecules and hydroxyl groups inside the sepiolite. Its characteristic absorption peaks typically appear at 3500-3700 nm. The range reflects the vibrational characteristics of hydroxyl groups. Furthermore, the silicon-oxygen tetrahedral and magnesium-oxygen octahedral structures in sepiolite exhibit unique absorption peaks in the infrared spectrum, further revealing its chemical structural characteristics. X-ray diffraction (XRD), a standard identification method in mineralogy, can meticulously depict the crystal structure of sepiolite. The XRD pattern of sepiolite exhibits a series of unique diffraction peaks, the distribution and intensity of which are closely related to its crystal structure. Using combined infrared spectroscopy and XRD analysis, sepiolite can be effectively distinguished from clay minerals such as montmorillonite and kaolinite. Scanning electron microscopy (SEM) can clearly display the fibrous or layered microstructure of sepiolite, while energy dispersive spectroscopy (EDS) can accurately determine the content of elements such as magnesium, silicon, and oxygen in sepiolite. Combining SEM and EDS analysis, we can comprehensively reveal the morphological characteristics and elemental composition of sepiolite, thereby obtaining multimodal data for identification. Multimodal data includes image data and spectral data rich in sepiolite. For a microscopic illustration of sepiolite, please refer to [link to relevant documentation]. Figure 2 ,like Figure 2 As shown, a) is pure sepiolite observed under natural light; b) is sepiolite under cross-polarized light; c) is continuous sepiolite fibers under a transmission electron microscope image; d) is a transmission electron microscope image showing the ordered arrangement of asbestos-like sepiolite; e) is a sepiolite aggregate; f) is fibrous sepiolite; g) is a pad formed by relatively long sepiolite fibers; and h) is a thick aggregate of sepiolite. For a schematic diagram of the XRD patterns of sepiolite, please refer to [link to schematic diagram]. Figure 3 Please refer to the infrared spectrum diagram of its sepiolite. Figure 4 ,like Figure 4 As shown, a is the infrared spectrum of pure sepiolite, b is the infrared spectrum of sepiolite after adsorbing methyl green, c is the infrared spectrum of sepiolite after adsorbing crystal violet, and d is the infrared spectrum of sepiolite after adsorbing methylene blue.

[0076] In some embodiments, in step S102, feature extraction is performed on the multimodal data based on the feature extraction layer to obtain image feature vectors and text feature vectors. The hybrid neural network model includes a ResNet model, a 3D-CNN network, and an LSTM network. First, the image data and spectral data in the multimodal data are preprocessed to obtain a rock image dataset. Targeted preprocessing methods are used for different types of data to improve data quality. For the acquired image data, Gaussian filtering is used to denoise the image data to eliminate noise interference. Histogram equalization is then used to enhance the denoised image data, i.e., to enhance the contrast and clarity of the image. Finally, the enhanced image data is normalized, i.e., the image data is normalized to... The image data is preprocessed by dividing the image into intervals. For spectral data, a peak detection algorithm (local maximum method) is used to identify the characteristic peaks and calculate their integral areas to obtain a set of characteristic peaks related to sepiolite. Specifically, characteristic peaks related to sepiolite are selected, irrelevant peaks are removed, and dynamic time warping is used to align the spectral data sequence using the characteristic peak set as the reference anchor point. This ensures that the characteristic peak positions of all spectral data are consistent. The aligned spectral data is then truncated to maintain consistent spectral lengths, completing the spectral data preprocessing. The preprocessed image data and spectral data are then merged and processed. The merged data was processed using multivariate scattering correction and standard normal transformation to obtain a rock sample dataset, which included image data, spectral data, and geological attributes. After preprocessing the image and spectral data, the image and spectral data were merged. Multivariate scattering correction was then used to correct spectral differences caused by scattering effects, highlighting the characteristic absorption peaks of sepiolite. Standard normal transformation was used to eliminate the influence of factors such as sample concentration and particle size on spectral intensity. At the same time, smoothing filtering and wavelet transform were used to reduce noise and improve the signal-to-noise ratio of the spectrum, further optimizing the data quality.

[0077] Secondly, image feature extraction is performed on the rock sample dataset based on the ResNet model to obtain image feature vectors. In the hybrid neural network model, ResNet is used as the core architecture for image feature extraction. The image data from the rock sample dataset is input into the ResNet model and processed layer by layer through multiple convolution, pooling, and residual modules. The convolutional layers use convolutional kernels of various sizes to slide on the image, extracting local details such as edges and textures, gradually transitioning from low-level features to high-level features. Then, the pooling layers downsample the convolution results to reduce the computational burden and simplify the model. Subsequently, the residual module adds the input and convolution output through skip connections, effectively alleviating the computational burden in deep networks. To address the vanishing and exploding gradient problems, global average pooling compresses the feature map into a fixed-length vector, which is then passed through a fully connected layer to output the classification result, thus obtaining the image feature vector. The model can extract high-level features from the image in the deep structure. These high-level features can accurately capture the semantic information in the image, including key elements such as the shape, color, and texture of objects. In particular, the unique porous honeycomb structure of sepiolite provides an important basis for distinguishing sepiolite from other rock types. ResNet can effectively capture the spatial features of sepiolite through multi-layer convolution and pooling structures. At the same time, its unique shortcut connection mechanism avoids the vanishing gradient problem, ensuring the stability and efficiency of deep network training.Finally, a 3D-CNN network is used to perform convolution operations on the rock sample dataset to obtain feature maps that fuse spatiotemporal information. An LSTM network is then used to extract features from these feature maps to obtain text feature vectors. For the IR, XRD, SEM, and EDS serialized data in the spectral data, these data are input into an LSTM network. Combined with a 3D convolutional neural network (3D-CNN), the dynamic characteristics of sepiolite changing over time are captured. LSTM, with its input gate, forget gate, and output gate gating mechanism, can effectively control information transmission, accurately capture long-distance dependencies in the text, and generate fixed-dimensional text feature vectors. Specifically, a Transformer-based bidirectional encoding model (BERT) generates context-dependent word vectors, which are then input into a 3D convolutional neural network (3D-CNN). In CNN and LSTM networks, to capture the dynamic characteristics of sepiolite's evolution over time, 3D convolution treats text as a sequence containing time and feature dimensions. By sliding convolution kernels across multiple channels, convolution operations are performed on neighboring words and their features to generate feature maps that fuse spatiotemporal information. After receiving the feature maps, LSTM processes the information using memory units and gating mechanisms. Memory units store short-term and long-term information, while gating mechanisms regulate the flow of information. LSTM reads the feature maps step by step over time, filters key information through gating, deeply mines the semantics and contextual relationships of the text, and integrates these semantics and contextual relationships into word vectors. Finally, the text feature vector is output through a fully connected layer and processed by LSTM. LSTM, with its gating structure, can effectively capture long-term dependencies and extract key features from the graph, thereby helping the model to deeply understand the crystal structure and composition information of sepiolite.

[0078] In some embodiments, in step S103, the image feature vector and text feature vector are concatenated based on the early fusion layer to obtain a multimodal feature vector. The ResNet model includes multi-layer convolutional modules, pooling modules, and residual modules. The early fusion layer concatenates the image feature vector and text feature vector to obtain a multimodal feature vector. Specifically, the image feature vector is locally feature-extracted through the multi-layer convolutional modules of the ResNet model to obtain image features, and the text feature vector is feature-extracted through the LSTM network to obtain text features. The image features and text features are then concatenated to obtain a multimodal feature vector. The early fusion layer is part of the hybrid neural network model. In the shallow fusion model, during the data input stage, feature vectors from the image, spectrum, and geological background are stitched together to merge the pixel matrix and numerical features according to rules, generating fused input data. Subsequently, the fused input data is fed into a unified network structure containing ResNet and LSTM, i.e., the early fusion layer. In the early fusion layer, the convolutional layers of the ResNet model are responsible for extracting local features of the image, while the LSTM network uses its sequence processing capabilities to capture time series or contextual information. During training, the entire network is optimized in an end-to-end manner, prompting ResNet and LSTM to work together and learn feature representations that help identify sepiolite.

[0079] In some embodiments, in step S104, the multimodal feature vectors are weighted and fused based on the intermediate fusion layer to obtain a fused feature vector. The multimodal feature vectors are then input into the intermediate fusion layer. The intermediate fusion layer fuses the image feature vectors and text feature vectors from the multimodal feature vectors through weighted fusion. Specifically, the image feature vectors of the multimodal feature vectors are convolutionally and pooled using the convolution and pooling modules of the ResNet model to obtain image features. The text feature vectors of the multimodal feature vectors are extracted using an LSTM network to obtain text features. The image features and text features are then fused through weighted fusion to obtain a fused feature vector. In the intermediate layer of the hybrid neural network model, the rock data is first fed into separate ResNet and LSTM networks for feature extraction. ResNet uses multi-layer convolution and pooling operations to extract intermediate features such as edges and textures from the rock images. LSTM processes the sequence data and mines time-dependent or context-dependent features. Then, in the intermediate layer of the network, the features extracted by ResNet and LSTM are combined through weighted fusion. Weights are assigned to different features according to task requirements. The fused features are then input into the subsequent late-stage fusion layer for further learning and feature transformation. Finally, the classifier outputs the sepiolite identification result. During training, the entire network participates in optimization, adjusting parameters to improve recognition performance.

[0080] In some embodiments, in step S105, classification features are extracted from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features. The image classification features and text classification features are then fused. The fused classification features, the fused feature vector, and the multimodal feature vector are input into the classifier to identify sepiolite in rocks. In the final fusion stage of the hybrid neural network model, the late fusion layer extracts classification features from the fused feature vector based on the ResNet model to obtain image classification features; it also extracts classification features from the fused feature vector based on the LSTM network to obtain text classification features; a weighted average method is used to fuse the image classification features and text classification features; and a fully connected layer is used to perform feature pairing between the fused feature vector and the multimodal feature vector and the fused classification features. After alignment, the data is input into the Softmax classifier to identify sepiolite in the rock. The fused feature vector is then input into the late fusion layer. ResNet and LSTM independently perform forward propagation calculations on the rock data. ResNet processes the image data through convolution, pooling, and residual modules to generate feature representations for classification. LSTM processes the sequence data, using memory units to store short-term and long-term information and regulating information flow through a gating mechanism to finally output classification features. Subsequently, a weighted average method is used to fuse the output features of ResNet and LSTM. After aligning the multimodal feature vectors of the early fusion layer and the fused feature vectors of the mid-term fusion layer with the fused classification features of the late fusion layer through a fully connected layer, the data is input into the Softmax classifier to determine the presence of sepiolite in the rock.

[0081] A hybrid neural network model was trained based on a rock image dataset. Feature extraction was performed on the rock image dataset to obtain image and text feature vectors. These feature vectors were then fused using early, mid, and late fusion layers of the hybrid neural network model to identify sepiolite. After preprocessing the spectral and image data, core features significantly contributing to sepiolite identification were selected, including spatial, spectral, geometric, and temporal features. A multi-layered deep learning framework with early, mid, and late fusion layers in the hybrid neural network model was used to achieve centralized processing of multi-source data and complementary feature advantages, thereby significantly improving the model's recognition and discrimination capabilities. The training process of the hybrid neural network model involved dividing the rock sample dataset into training, validation, and test sets with proportions of 70%, 15%, and 15%, respectively. The training and validation sets were input into the hybrid neural network model. The model was trained using the training set, and its hyperparameters were adjusted using the validation set to obtain a fully trained hybrid neural network model. Finally, the performance of the fully trained hybrid neural network model was evaluated using the test set.

[0082] In the feature fusion stage of the hybrid neural network model (early fusion layer, mid-term fusion layer, and late fusion layer), the image features extracted by ResNet are combined with the sequence features processed by LSTM. A feature splicing fusion strategy is adopted, which enables the model to utilize the dual information of image and sequence data at the same time. Through the dual information, the model can accurately identify the feature patterns related to sepiolite in the image, thereby improving the accuracy and efficiency of sepiolite identification.

[0083] During the model training phase, appropriate loss functions were selected for different tasks. For the binary classification task of determining the presence or absence of sepiolite, the cross-entropy loss function was used; while for the regression task of predicting sepiolite content, the mean squared error loss function was used. The model was trained iteratively multiple times using the Adam high-efficiency optimizer based on the training data, and the model performance was monitored in real time using the validation set to adjust hyperparameters promptly to prevent overfitting. For a comparison diagram of the Adam optimizer, please refer to [link / reference needed]. Figure 5 ,like Figure 5 The image shows a comparison of different learning rates and weight decay parameters for the Adam optimizer. The horizontal axis represents the number of training iterations of the hybrid neural network model, and the vertical axis represents the loss value of the hybrid neural network model. Training stops when the loss value falls below 0.1%. Indicates the learning rate. (representing the weight decay parameter). On the test set, a fully trained hybrid neural network model is used to predict the test set, and the prediction results are obtained. The prediction results are evaluated by accuracy, recall, and mean squared error. The model is comprehensively evaluated, focusing on its performance in sepiolite identification, content prediction, and crystal structure analysis. Continuous optimization is performed to bring the model to its optimal state, thereby ensuring accurate identification of sepiolite in rocks. The loss function is:

[0084] ,

[0085] ,

[0086] in, Let cross-entropy be the loss function. Let the mean squared error loss function be . This represents the probability distribution of the true labels. The probability distribution predicted by the model. For samples or events, For the sample size, For the first The true value of each sample For the first The predicted value for each sample.

[0087] To comprehensively evaluate model performance, multiple metrics were used, including recall, F1 score, and area under the ROC curve (AUC). For a schematic diagram of the ROC curve, please refer to [link to ROC curve diagram]. Figure 6 By employing cross-validation, the impact of data partitioning on the evaluation results was effectively reduced. Key metrics from the model training process are recorded in Table 2.

[0088] Table 2

[0089]

[0090] Based on the evaluation metrics, the model parameters were carefully adjusted and the overall strategy was optimized, which significantly improved the model's performance in identifying sepiolite in rocks. In addition, according to the needs of actual application scenarios, the number of ResNet layers, convolution kernel size, and the number of hidden layers and neurons of LSTM were flexibly adjusted to adapt to recognition tasks of different dataset sizes and complexities.

[0091] Based on a well-trained hybrid neural network model, the model predicts the rock samples to be predicted in order to identify sepiolite in the rocks. The accuracy rate of identifying high-purity sepiolite in the rock samples reached 98%, medium-purity sepiolite 92%, and low-purity sepiolite 90%.

[0092] In summary, the sepiolite identification method in rocks provided by this invention is used to identify sepiolite in rocks based on a well-trained hybrid neural network model. The hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer. Based on the feature extraction layer, features are extracted from multimodal data to obtain image feature vectors and text feature vectors. Based on the early fusion layer, the image feature vectors and text feature vectors are concatenated to obtain multimodal feature vectors. Based on the mid-term fusion layer, the multimodal feature vectors are weighted and fused to obtain fused feature vectors. Based on the late fusion layer, classification features are extracted from the fused feature vectors to obtain image classification features and text classification features. The image classification features and text classification features are fused. The fused classification features, the fused feature vectors, and the multimodal feature vectors are input into a classifier to identify sepiolite in rocks, thereby improving the identification efficiency and accuracy of sepiolite in rocks.

[0093] To better implement the method for identifying sepiolite in rocks in this embodiment of the invention, based on the method for identifying sepiolite in rocks, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a device for identifying sepiolite in rocks, used to identify sepiolite in rocks based on a fully trained hybrid neural network model. The hybrid neural network model includes a feature extraction layer, an early fusion layer, a mid-term fusion layer, and a late fusion layer. The sepiolite identification device 700 in rocks includes:

[0094] The data acquisition module 701 is used to acquire the multimodal data to be identified, including sepiolite-rich image data and spectral data;

[0095] The feature extraction module 702 is used to extract features from multimodal data based on the feature extraction layer to obtain image feature vectors and text feature vectors;

[0096] The early fusion module 703 is used to concatenate image feature vectors and text feature vectors based on the early fusion layer to obtain multimodal feature vectors;

[0097] The intermediate fusion module 704 is used to perform weighted fusion of multimodal feature vectors based on the intermediate fusion layer to obtain the fused feature vector;

[0098] The late fusion module 705 is used to extract classification features from the fused feature vector based on the late fusion layer to obtain image classification features and text classification features. The image classification features and text classification features are fused, and the fused classification features, fused feature vectors and multimodal feature vectors are input into the classifier to identify sepiolite in rocks.

[0099] like Figure 8As shown, the present invention also provides an electronic device 800, which can be a mobile terminal, desktop computer, laptop, handheld computer, server, or other computing device. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0100] In some embodiments, memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or memory of the electronic device 800. In other embodiments, memory 802 may be an external storage device of the electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 800. Furthermore, memory 802 may include both internal and external storage units of the electronic device 800. Memory 802 is used to store application software and various types of data installed on the electronic device 800, such as program code installed on the electronic device 800. Memory 802 may also be used to temporarily store data that has been output or will be output. In one embodiment, memory 802 stores a program for identifying sepiolite in rocks, which can be executed by processor 801 to implement the sepiolite identification method in rocks according to various embodiments of the present invention.

[0101] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor or other data processing chip, used to run program code stored in memory 802 or process data, such as a method for identifying sepiolite in rocks.

[0102] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display identification information from a sepiolite identification procedure in rocks and to display a visual user interface. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0103] In some embodiments, when the processor 801 executes the sepiolite identification program in the memory 802, it implements each step of the sepiolite identification method in the rock as described in the above embodiments. Since the sepiolite identification method in the rock has been described in detail above, it will not be repeated here.

[0104] Accordingly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps or functions of the methods for identifying sepiolite in rocks provided in the above-described method embodiments.

[0105] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0106] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying sepiolite in a rock, characterized by, The application relates to a method for identifying sepiolite in rock based on a trained complete hybrid neural network model, wherein the hybrid neural network model comprises a feature extraction layer, an early fusion layer, a middle fusion layer and a late fusion layer. The method for identifying sepiolite in rock comprises the following steps: Multi-dimensional analysis is performed on a rock sample to obtain multi-modal data to be identified, wherein the multi-modal data comprises image data and spectrum data rich in sepiolite; the multi-dimensional analysis of the rock sample comprises infrared spectrum analysis for detecting the vibration behavior of water molecules and hydroxyl groups in the sepiolite, X-ray diffraction technology for depicting the crystal structure of the sepiolite, and scanning electron microscopy and energy spectrum analysis for revealing the morphological characteristics and element composition of the sepiolite; Feature extraction is performed on the multi-modal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; The hybrid neural network model comprises a ResNet model, a 3D-CNN network and an LSTM network; the feature extraction on the multi-modal data based on the feature extraction layer to obtain the image feature vector and the text feature vector comprises the following steps: The multi-modal data is preprocessed to obtain a rock sample data set; Image feature extraction is performed on the rock sample data set based on the ResNet model to obtain an image feature vector; Convolution operation is performed on the rock sample data set based on the 3D-CNN network to obtain a feature map with fused space-time information, and feature extraction is performed on the feature map with fused space-time information based on the LSTM network to obtain a text feature vector; The preprocessing of the multi-modal data to obtain a rock sample data set comprises the following steps: Gaussian filtering is adopted to perform denoising processing on the image data, histogram equalization is adopted to perform image enhancement on the image data after the denoising processing, and normalization processing is performed on the image data after the image enhancement, so as to complete the preprocessing of the image data; A peak detection algorithm is adopted to identify the characteristic peaks of the spectrum data, and the integral areas of the characteristic peaks are calculated to obtain a characteristic peak set related to sepiolite; Dynamic time warping is adopted to perform sequence alignment on the spectrum data with the characteristic peak set as a reference anchor point, and the spectrum data after the sequence alignment is truncated to complete the preprocessing of the spectrum data; The preprocessed image data and spectrum data are merged, and the merged data is processed by multivariate scatter correction and standard normal variation to obtain a rock sample data set; The image feature vector and the text feature vector are spliced based on the early fusion layer to obtain a multi-modal feature vector; The multi-modal feature vector is weighted and fused based on the middle fusion layer to obtain a fused feature vector; Classification feature extraction is performed on the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, the image classification features and the text classification features are fused, the fused classification features, the fused feature vector and the multi-modal feature vector are input into a classifier, and sepiolite in rock is identified.

2. The method for identifying sepiolite in rock according to claim 1, characterized in that, The ResNet model comprises a plurality of convolution modules, a pooling module, and a residual module; the image feature vector and the text feature vector are spliced based on the early fusion layer to obtain a multi-modal feature vector, which comprises: The image feature is extracted from the image feature vector by the plurality of convolution modules of the ResNet model to obtain an image feature; The text feature is extracted from the text feature vector by the LSTM network to obtain a text feature; The image feature and the text feature are spliced to obtain a multi-modal feature vector.

3. The method for identifying sepiolite in rock according to claim 2, characterized in that, The multi-modal feature vector is weighted fused based on the mid-fusion layer to obtain a fused feature vector, which comprises: The image feature vector of the multi-modal feature vector is subjected to convolution and pooling operations by the convolution module and the pooling module of the ResNet model to obtain an image feature; The text feature of the multi-modal feature vector is extracted by the LSTM network to obtain a text feature; The image feature and the text feature are weighted fused to obtain a fused feature vector.

4. The method for identifying sepiolite in a rock according to claim 3, characterized in that, The fused feature vector is classified and extracted based on the late fusion layer to obtain an image classification feature and a text classification feature, and the fused classification feature, the fused feature vector, and the multi-modal feature vector are input into a classifier to identify the palygorskite in the rock, which comprises: The fused feature vector is classified and extracted based on the ResNet model to obtain an image classification feature; The fused feature vector is classified and extracted based on the LSTM network to obtain a text classification feature; The image classification feature and the text classification feature are fused by a weighted average method, and the fused feature vector and the multi-modal feature vector are aligned with the fused classification feature by a full connection layer, and then input into a Softmax classifier to identify the palygorskite in the rock.

5. The method for identifying sepiolite in rock according to claim 4, characterized in that, The loss function of the hybrid neural network model is: ， ， where, is a cross-entropy loss function, is a mean squared error loss function, is a probability distribution of true labels, is a probability distribution predicted by a model, is a sample or event, is a number of samples, is a true value of the th sample, is a predicted value of the th sample.

6. An apparatus for identifying sepiolite in a rock, characterized by The hybrid neural network model is used for identifying the palygorskite in the rock based on a trained complete hybrid neural network model, which comprises a feature extraction layer, an early fusion layer, a mid-fusion layer, and a late fusion layer. The device for identifying the palygorskite in the rock comprises: A data acquisition module is configured to perform multi-dimensional analysis on a rock sample to obtain multi-modal data to be identified, wherein the multi-modal data comprises image data and spectral data rich in palygorskite; the multi-dimensional analysis on the rock sample comprises infrared spectrum analysis for detecting the vibration behavior of water molecules and hydroxyl groups in the palygorskite, X-ray diffraction technology for depicting the crystal structure of the palygorskite, and scanning electron microscopy and energy spectrum analysis for revealing the morphological characteristics and element composition of the palygorskite; A feature extraction module is configured to extract features from the multi-modal data based on the feature extraction layer to obtain an image feature vector and a text feature vector; The mixed neural network model comprises a ResNet model, a 3D-CNN network, and an LSTM network; the feature extraction layer is used to extract features of the multi-modal data to obtain an image feature vector and a text feature vector, which comprises: The multi-modal data is preprocessed to obtain a rock sample data set; The ResNet model is used to extract image features of the rock sample data set to obtain an image feature vector; The 3D-CNN network is used to perform convolution operation on the rock sample data set to obtain a feature map with fused space-time information, and the LSTM network is used to extract features of the feature map with fused space-time information to obtain a text feature vector; The multi-modal data is preprocessed to obtain a rock sample data set, which comprises: Gaussian filtering is used to denoise the image data, histogram equalization is used to enhance the denoised image data, and the enhanced image data is normalized to complete the preprocessing of the image data; A peak detection algorithm is used to identify the characteristic peaks of the spectral data, and the integral areas of the characteristic peaks are calculated to obtain a characteristic peak set related to sepiolite; The spectral data is sequentially aligned by using dynamic time warping based on the characteristic peak set as the reference anchor point, and the sequentially aligned spectral data is truncated to complete the preprocessing of the spectral data; The preprocessed image data and spectral data are merged, and the merged data is processed by multivariate scatter correction and standard normal variation to obtain a rock sample data set; An early fusion module is configured to splice the image feature vector and the text feature vector based on the early fusion layer to obtain a multi-modal feature vector; A middle fusion module is configured to perform weighted fusion on the multi-modal feature vector based on the middle fusion layer to obtain a fused feature vector; A late fusion module is configured to perform classification feature extraction on the fused feature vector based on the late fusion layer to obtain image classification features and text classification features, fuse the image classification features and the text classification features, input the fused classification features, the fused feature vector, and the multi-modal feature vector into a classifier, and identify sepiolite in the rock.

7. An electronic device, comprising: The device comprises a memory and a processor; The memory stores a computer readable program that can be executed by the processor; The processor executes the computer readable program to implement the steps of the method for identifying sepiolite in the rock according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The device stores a computer readable program or instructions that can be executed by a processor to implement the steps of the method for identifying sepiolite in the rock according to any one of claims 1-5.

Images