A rock zero-shot classification method, device and medium

Abstract

The application discloses a rock zero-sample classification method, equipment and medium, and the method comprises the following steps: acquiring rock images of multiple scenes and text semantic descriptions corresponding to each rock image, performing standardization preprocessing on each rock image, and generating standardized training data; taking a CLIP visual-language pre-training architecture as a base, embedding a rock microtexture dynamic enhancement module to strengthen microtexture feature extraction, realizing accurate matching of image-text multi-granularity semantics through a cross-modal multi-granularity semantic self-adaptive alignment mechanism, combining a domain generalization visual prototype robust distillation method to compress intra-class differences and eliminate scene deviation, adopting two-stage training to complete optimization of a classification model, inputting rock images to be classified into the final classification model, and outputting a category classification result after feature extraction and cross-modal semantic matching calculation. The application can realize high-precision zero-sample identification of unseen rock categories in a complex field environment, and effectively improves the accuracy and robustness of rock classification in geological exploration.

Description

Technical Field

[0001] This application relates to the fields of computer vision and geological exploration technology, and in particular to a method, device and medium for zero-sample rock classification. Background Technology

[0002] With the transformation of geological exploration and geotechnical engineering towards intelligence, computer vision technology is being widely used in automated rock classification.

[0003] Existing rock classification methods are mostly based on pure visual models such as convolutional neural networks. While they achieve high accuracy on closed datasets, they are essentially closed-set recognition paradigms, requiring supervised training with a large number of labeled samples. However, there are many types of rocks in nature, and newly discovered samples in the field often do not appear in the training set, causing the model to completely fail for unseen categories and lacking zero-shot generalization ability. At the same time, field-collected images are often accompanied by interference such as changes in lighting, surface dirt, and occlusion, further increasing the difficulty of recognition.

[0004] Although large-scale visual-language pre-trained models such as CLIP have been introduced into rock classification to achieve zero-sample transfer through textual semantic guidance, rock classification is highly dependent on fine-grained texture features such as crystal grains and bedding structures. However, the CLIP image encoder uses a large block strategy, which easily leads to the aliasing and loss of high-frequency texture information, resulting in a decrease in the discriminative power of rock subclasses that are similar in appearance but different in microstructure. Furthermore, CLIP aligns image features at a single scale with complex textual semantics that include both macroscopic morphology and microstructure, and cannot dynamically adjust the focus of the image encoder according to the granular level emphasized in the text, resulting in mismatch between image and textual features. Summary of the Invention

[0005] To address the technical problems existing in the background art, embodiments of this application provide a rock zero-shot classification method, device, and medium. The method includes: acquiring rock images from multiple scenes and corresponding textual semantic descriptions for each rock image; performing standardized preprocessing on each rock image to generate standardized training data; constructing a rock zero-shot classification model based on the CLIP vision-language pre-training architecture; the rock zero-shot classification model sequentially embedding a rock micro-texture dynamic enhancement module, a cross-modal multi-granularity semantic adaptive alignment mechanism, and a domain-generalized visual prototype robust distillation method; the rock micro-texture dynamic enhancement module utilizes dynamic convolution and dual-branch... Attention-enhanced rock micro-texture feature extraction; the cross-modal multi-granularity semantic adaptive alignment mechanism achieves adaptive alignment of rock image multi-granularity features with text semantics through multi-granularity feature fusion and text semantic guidance; the domain generalization visual prototype robust distillation method compresses intra-class differences and eliminates scene offset through visual prototype constraints and domain adversarial training; the rock zero-shot classification model is trained in two stages using the standardized training data to obtain the final rock zero-shot classification model; the rock image to be classified is input into the final rock zero-shot classification model, and after feature extraction and cross-modal semantic matching calculation, the corresponding rock category classification result is output.

[0006] In one example, a dynamic enhancement module for rock microtexture in a zero-shot rock classification model is constructed based on the CLIP vision-language pre-trained architecture. Specifically, this includes: using the CLIP-ViT-Base-Patch16 pre-trained model as a foundation, retaining the complete pre-trained parameters and network structure of the image encoder and text encoder; and in the 5th, 6th, and 7th layers of the image encoder's Transformer... The output positions of the encoder are sequentially inserted into the dynamic enhancement module for rock micro-texture. This module includes a depthwise convolution kernel generation unit, which uses an Unpatchify operation to reverse-map the input PatchEmbedding into a two-dimensional feature map. After parallel convolutional layers and global average pooling, a depthwise convolution kernel corresponding to each feature position is generated. A dual-branch channel attention mask generation unit is also included, extracting channel dimension weights and spatial location correlation weights respectively, and fusing them to generate a normalized channel attention mask. A texture feature reconstruction and enhancement unit is also included, performing depthwise separable convolution through the depthwise convolution kernels to extract micro-texture features. Based on the micro-texture features and the channel attention mask, the graphics encoder features and texture enhancement features are weighted and fused, converted into a Patch Embedding of the corresponding dimension, and output to the next layer of the graphics encoder.

[0007] In one example, based on the CLIP vision-language pre-training architecture, a cross-modal multi-granularity semantic adaptive alignment mechanism for a rock zero-shot classification model is constructed. Specifically, this includes: inserting the cross-modal multi-granularity semantic adaptive alignment mechanism at the output of the 12th layer of the Transformer Encoder in the image encoder; the cross-modal multi-granularity semantic adaptive alignment mechanism includes extracting the Class Tokens output from the 5th, 8th, and 12th layers of the image encoder, concatenating them with channels via Unpatchify to obtain multi-granularity feature maps corresponding to shallow edge contours, mid-layer surface textures, and deep overall structures; constructing a text semantic-guided gating network, generating cross-modal global features based on the mean, variance, and extreme value features of all categories of text features, and outputting adaptive weights for the three granularities of features via a weighted network; and constructing a multi-granularity feature adaptive asymmetric fusion unit, which weights and progressively fuses the multi-granularity feature maps according to the adaptive weights to generate image feature vectors.

[0008] In one example, a domain-generalized visual prototype robust distillation method for constructing a rock zero-shot classification model is built based on the CLIP vision-language pre-training architecture. Specifically, this method includes: embedding a domain-generalized visual prototype robust distillation method; the domain-generalized visual prototype robust distillation method extracts image feature vectors from rock images of each known category using a CLIP-ViT-Base-Patch16 pre-trained model, performs K-Means clustering on all feature vectors of each rock category; uses the mean of the clustered vectors as the visual prototype of the rock category, and the visual prototypes of all categories together constitute a visual prototype library. A domain classifier is connected after the image encoder, and a gradient inversion layer is introduced between the image encoder and the domain classifier to construct the rock zero-shot classification model.

[0009] In one example, the rock zero-shot classification model is initially trained using the standardized training data. Specifically, this includes: freezing all parameters of the image encoder and text encoder in the CLIP-ViT-Base-Patch16 base model, iteratively optimizing the learnable parameters of the rock micro-texture dynamic enhancement module and the cross-modal multi-granularity semantic adaptive alignment mechanism, and obtaining the initial rock zero-shot classification model.

[0010] In one example, the primary rock zero-shot classification model is jointly fine-tuned end-to-end using the standardized training data. Specifically, this includes: unfreezing all parameters of the image encoder and text encoder in the CLIP-ViT-Base-Patch16 base model; introducing intra-class center loss and domain adversarial loss, which together with the CLIP native visual-language contrastive learning loss constitute the total loss function to optimize the model parameters of the primary rock zero-shot classification model, thus obtaining the final rock zero-shot classification model.

[0011] In one example, the rock image to be classified is input into the final rock zero-shot classification model. After feature extraction and cross-modal semantic matching calculation, the corresponding rock category classification result is output. Specifically, this includes: inputting the rock image to be classified into the final rock zero-shot classification model to obtain the feature vector of the rock image to be classified; mapping the feature vector to the feature manifold defined by the visual prototype library, calculating the attention weights of all visual prototypes corresponding to the feature vector, and obtaining the reconstructed projection vector by weighted summation; calculating the Mahalanobis distance between the reconstructed projection vector and the text feature vectors of each rock category, and selecting the category with the smallest Mahalanobis distance as the rock classification result of the rock image to be classified.

[0012] In one example, each rock image undergoes standardized preprocessing to generate standardized training data. Specifically, this includes: performing resolution unification and color space normalization on each rock image; performing automatic white balance and denoising on each normalized rock image in sequence; enhancing the contrast of each denoised rock image through histogram equalization; and performing data augmentation on each enhanced rock image to obtain standardized training data. The data augmentation includes random rotation, horizontal flipping, and brightness jitter.

[0013] On the other hand, embodiments of this application provide a rock zero-sample classification device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-mentioned rock zero-sample classification methods.

[0014] On the other hand, embodiments of this application provide a non-volatile computer storage medium for rock zero-sample classification, which stores computer-executable instructions that can execute any of the above-mentioned rock zero-sample classification methods.

[0015] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: This invention is based on the CLIP vision-language pre-training architecture. By introducing a dynamic enhancement module for rock micro-texture, it effectively repairs and enhances fine-grained texture features such as rock crystal grain size and layering structure using dynamic convolution and a bi-branch attention mechanism, significantly improving the model's ability to distinguish rock subclasses with similar textures but different microstructures. Through a cross-modal multi-granularity semantic adaptive alignment mechanism, it achieves adaptive alignment between multi-granular features of images and macro- and micro-textual semantics of text, solving the image-text misalignment problem caused by single-scale feature matching in traditional CLIP models. By combining a domain generalization visual prototype robust distillation method with visual prototype constraints and domain adversarial training, it effectively compresses intra-class differences and eliminates domain offset between laboratory and field scenes.

[0016] The model in this scheme can identify entirely new rock categories without retraining. It significantly improves the classification accuracy and robustness of unseen rock categories in complex field environments, effectively meeting the practical needs of geological exploration for high-precision and highly adaptable rock classification. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, some embodiments of this application will be described in detail below with reference to the accompanying drawings, in which: Figure 1 A flowchart illustrating a rock zero-sample classification method provided in this application embodiment; Figure 2 A diagram illustrating the architecture of a zero-sample rock classification model provided in this application embodiment; Figure 3 A flowchart of a rock microtexture dynamic enhancement module for a rock zero-sample classification method provided in this application embodiment; Figure 4 A diagram illustrating a cross-modal, multi-granularity semantic adaptive alignment mechanism for a rock zero-sample classification method provided in this application embodiment; Figure 5 A flowchart of a domain generalization visual prototype robust distillation method for a rock zero-sample classification method provided in this application embodiment; Figure 6 This is a schematic diagram of the structure of a rock zero-sample classification device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] Some embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating a zero-sample rock classification method provided in this application embodiment. This method can be applied to different business domains. Certain input parameters or intermediate results in this process can be manually adjusted to help improve accuracy.

[0021] The analysis method involved in the embodiments of this application can be implemented by a terminal device or a server, and this application does not impose any special limitations on it. For ease of understanding and description, the following embodiments are all described in detail using a server as an example.

[0022] Based on this Figure 1 The process may include the following steps: S101: Obtain rock images from multiple scenes and the corresponding text semantic descriptions for each rock image. Perform standardized preprocessing on each rock image to generate standardized training data.

[0023] In some embodiments of this application, data acquisition encompasses both laboratory environments and real-world field scenarios to ensure the diversity and representativeness of the samples. The acquisition standards strictly adhere to geological exploration specifications. Laboratory samples must be selected from standard rock blocks that are clean, structurally intact, and clearly reflect typical geological formations, while field samples must include complex environmental interference factors such as variations in light intensity, weathered surfaces, and soil obstruction.

[0024] During the data collection process, high-resolution industrial cameras were used to capture images of laboratory rocks from multiple angles under fixed light conditions, while images of outcrops and cores in the field were also collected under natural light conditions. Geological labels for each rock image were recorded simultaneously. These geological labels included the geological age of the rock, its main mineral composition, and macroscopic structural features. The geological labels were then converted into text descriptions, and a multi-source image dataset containing various types of rocks, such as granite, basalt, and limestone, was finally constructed.

[0025] Furthermore, each rock image in the acquired multi-source image dataset underwent standardized preprocessing. The preprocessing first uniformly reset the resolution of each rock image to 224×224 pixels and performed color space normalization. Then, each normalized rock image underwent automatic white balance and denoising to correct color casts and blurring in field photography. Histogram equalization was used to enhance the contrast of each denoised rock image, highlighting the texture details of the rock surface. Finally, data augmentation was performed on each enhanced rock image, including random rotation, horizontal flipping, and brightness jittering, to expand the sample size. This process produced a batch of accurately labeled and feature-rich standardized training data, effectively enhancing the model's adaptability to different imaging conditions.

[0026] S102: Based on the CLIP vision-language pre-training architecture, a zero-shot rock classification model is constructed. This model sequentially embeds a dynamic enhancement module for rock microtexture, a cross-modal multi-granularity semantic adaptive alignment mechanism, and a domain-generalized visual prototype robust distillation method. The dynamic enhancement module enhances rock microtexture feature extraction through dynamic convolution and bi-branch attention. The cross-modal multi-granularity semantic adaptive alignment mechanism achieves adaptive alignment between multi-granularity features of rock images and text semantics through multi-granularity feature fusion and text semantic guidance. The domain-generalized visual prototype robust distillation method compresses intra-class differences and eliminates scene offsets through visual prototype constraints and domain adversarial training.

[0027] In some embodiments of this application, a pre-trained CLIP-ViT-Base-Patch16 base model is loaded based on the CLIP framework. Modular integration is achieved through the PyTorch 2.0 deep learning framework. The pre-trained parameters and network structure of the base model's image encoder and text encoder are fully preserved. The model is endowed with zero-shot recognition capabilities based on its pre-trained general visual-semantic implicit knowledge base. On this basis, a dynamic enhancement module for rock micro-texture, a cross-modal multi-granularity semantic adaptive alignment mechanism, and a domain generalization visual prototype robust distillation method are embedded in sequence to complete the overall model architecture.

[0028] Specifically, a dynamic enhancement module for rock micro-texture is first inserted sequentially at the output positions of the 5th, 6th, and 7th layers of the Transformer Encoder in the image encoder. This module contains a depthwise convolution kernel generation unit, a dual-branch channel attention mask generation unit, and a texture feature reconstruction and enhancement unit. The depthwise convolution kernel generation unit processes the patch embedding input to the Transformer Encoder through the Unpatchify operation, and generates a 22×22 resolution two-dimensional feature map in reverse. After parallel 1×1 convolutional layers and global average pooling, a depthwise convolution kernel corresponding to each feature position is dynamically generated. Relying on the spatial invariance of the convolution kernel, the local high-frequency details of the rock surface are accurately captured.

[0029] Furthermore, the dual-branch channel attention mask generation unit extracts channel dimension weights through global average pooling and spatial location association weights through self-attention weight matrix. After fusing channel and spatial dual attention information, it generates a normalized channel attention mask, effectively shielding the interference of environmental noise.

[0030] Furthermore, the texture feature reconstruction and enhancement unit performs depthwise separable convolution through dynamically generated depthwise convolution kernels to extract microscopic texture features of the rock. Then, it combines channel attention masks to weightedly fuse the encoder features and texture enhancement features, ultimately converting them into corresponding dimension patch embeddings for output to the next layer encoder. This process injects enhanced texture details while preserving the original semantic information, correcting the fine-grained texture loss problem caused by the large block strategy.

[0031] After the texture enhancement module is deployed, a cross-modal multi-granularity semantic adaptive alignment mechanism is integrated at the output of the 12th layer of the Transformer Encoder in the image encoder. This mechanism extracts the Class Tokens output from the 5th, 8th, and 12th layers of the image encoder through a Hook function, and then concatenates them with the channels via Unpatchify to obtain multi-granularity feature maps of the corresponding shallow edge contours, mid-layer surface textures, and deep overall structures, thus constructing a multi-scale feature system that conforms to the feature pyramid principle. At the same time, a text semantic-guided gating network is constructed, which transforms all category text descriptions into text feature vectors through a text encoder. The mean, statistical variance, and element-level maxima of all text feature vectors are calculated to obtain anchor text feature vectors, semantic fluctuation boundary vectors, and feature response extremum vectors. These are then concatenated to generate cross-modal global features and input into a weighted network, which outputs adaptive weights for the three granularity features.

[0032] Then, through the multi-granularity feature adaptive asymmetric fusion unit, the multi-granularity feature map is dynamically weighted and fused step by step according to the adaptive weight to generate an image feature vector that simultaneously contains macroscopic morphology and microscopic texture semantics. This achieves adaptive alignment between the multi-granularity features of the image and the macro and microscopic semantics of the text, solving the problem of image-text misalignment caused by single-scale feature matching.

[0033] Finally, a robust distillation method for domain generalization visual prototypes is configured for the entire model. This method involves extracting image feature vectors from rock images of each known category based on the pre-trained base model, performing K-Means clustering on all feature vectors of each rock category, and taking the mean of the clustered vectors as the visual prototype of that rock category. The visual prototypes of all categories together constitute the visual prototype library.

[0034] Feature clustering and storage are performed using the Faiss vector retrieval library, supporting fast Euclidean and Mahalanobis distance calculations. A domain classifier composed of multilayer perceptrons is connected after the image encoder at the training end, and a gradient inversion layer is introduced between the image encoder and the domain classifier to form a domain adversarial computation branch. A noise-robust inference unit is built at the inference end, and intra-class difference compression and scene domain offset elimination are achieved by relying on visual prototype constraints and domain adversarial training. Thus, the construction of a complete reinforcement perception and domain adaptive rock zero-shot classification model is completed.

[0035] S103: The rock zero-sample classification model is trained in two stages using the standardized training data to obtain the final rock zero-sample classification model.

[0036] In some embodiments of this application, standardized training data is used to perform two-stage training on the constructed rock zero-sample classification model, enabling smooth integration of the newly added modules with the general knowledge base of the base model and fully activating the performance of each module. The entire training process is carried out on an NVIDIA H200 GPU cluster, with the software environment built on Python 3.10, PyTorch 2.0 framework, and CUDA 12.8 acceleration library. The optimizer used is AdamW, with the initial learning rate set to 5e-5, the weight decay coefficient set to 0.2, and the batch size set to 128, balancing training efficiency and gradient stability.

[0037] An additional lightweight online data augmentation strategy is overlaid in the DataLoader of the training loop, including random cropping with a scaling ratio of 0.8-1.0 and random horizontal flipping with a probability of 0.5, to simulate the slight changes in the field of view of the field image. At the same time, brightness, contrast and color dithering enhancement operations are strictly prohibited to avoid conflict with the optimization objectives of the domain adversarial module in the domain generalization visual prototype robust distillation method, and to ensure the effective learning of domain invariant features.

[0038] The first training phase is the module adaptation pre-training phase, also known as the initial training phase. In this phase, all parameters of the image encoder and text encoder in the CLIP-ViT-Base-Patch16 base model are frozen. Only the learnable parameters in the rock micro-texture dynamic enhancement module and the cross-modal multi-granularity semantic adaptive alignment mechanism are iteratively optimized. The entire set of standardized training data is used as input, and the optimizer only updates the weights of the newly added modules. The training lasts for 10 epochs, allowing the new modules to complete adaptive adjustments in a stable pre-training semantic space, thus avoiding the destruction of the general knowledge of the base model. After completing this training phase, a primary rock zero-shot classification model is obtained.

[0039] The second training phase is the end-to-end joint fine-tuning phase, which unfreezes all trainable parameters of the model, enables the domain generalization visual prototype robust distillation method for joint optimization, introduces intra-class center loss and domain adversarial loss, and together with CLIP native visual-language contrastive learning loss, constitutes the total loss function.

[0040] The visual-language contrastive learning loss is used to narrow the feature distance between rock images and their corresponding text descriptions, while pushing away the feature distance between irrelevant text descriptions. The intra-class center loss is used to minimize the Euclidean distance between the intra-class image feature vector and the corresponding visual prototype, compressing the intra-class variance and forcing rock images of the same class to be closely distributed in the feature space, thereby enhancing the model's tolerance to different morphological variations of the same rock. The domain adversarial loss, combined with the gradient reversal layer, forces the image encoder to learn domain-invariant feature representations that cannot be distinguished by the domain classifier by flipping the gradient direction during backpropagation, effectively eliminating the distribution differences between laboratory data and field data.

[0041] To ensure stability in the early stages of training, a delayed start strategy is implemented for the domain adversarial loss: the weight coefficient of the domain adversarial loss is set to 0 in the first 3 epochs, and only the visual-language contrastive learning loss and intra-class center loss are optimized; from the 4th epoch onwards, the weight coefficient of the domain adversarial loss increases linearly from 0 to 0.5 with the number of training steps, gradually guiding the image encoder to learn domain-invariant features. After all training iterations are completed, the final rock zero-sample classification model is obtained.

[0042] S104: Input the rock image to be classified into the final rock zero-sample classification model. After feature extraction and cross-modal semantic matching calculation, the corresponding rock category classification result is output.

[0043] In some embodiments of this application, for the rock image to be identified, a standardization preprocessing operation is first performed to uniformly reset the resolution of the rock image to be classified to 224×224 pixels, and then automatic white balance correction, noise suppression and histogram equalization contrast enhancement are performed in sequence to ensure that the image format of the inference input is consistent with the distribution of the training data and to avoid feature extraction deviation caused by differences in input specifications.

[0044] After preprocessing, the image is input into the trained final rock zero-sample classification model. The image is first processed by an image encoder with an embedded rock micro-texture dynamic enhancement module for layer-by-layer feature extraction. At the 5th, 6th and 7th layers of the Transformer Encoder, the micro-texture features are enhanced and reconstructed by depthwise dynamic convolution and bi-branch attention masking, which fully preserves fine-grained recognition information such as rock crystal grain size and inter-crystal cementation relationship, and suppresses the interference of background and environmental noise.

[0045] The features are then transmitted to the cross-modal multi-granularity semantic adaptive alignment mechanism at the back end of the 12th layer encoder. The Hook function retrieves features from different depth levels. The gating network guided by text semantics dynamically adjusts the fusion weights of three granular features: shallow edge contours, mid-layer surface textures, and deep overall structure. The multi-granularity feature adaptive asymmetric fusion unit completes the step-by-step fusion, generating a high-quality image feature vector that simultaneously covers macroscopic morphology and microscopic structural semantics, thus achieving text-driven visual attention adaptation.

[0046] After obtaining the feature vector of the image to be classified, the model does not directly compare it with the text feature vector using conventional cosine similarity. Instead, it performs robust matching through the noisy robust inference module of the domain generalization visual prototype robust distillation method: First, the feature vector of the image to be classified is mapped to the feature manifold defined by the visual prototype library. Based on the similarity between the feature to be classified and the visual prototypes of each category, the attention weights of all corresponding visual prototypes are calculated. The reconstructed projection vector of the rock image feature vector in the feature space is generated by weighted summation. The prior distribution of the visual prototypes is used to eliminate the random noise interference of single sample features and compress the matching error caused by intra-class discreteness.

[0047] Subsequently, the Mahalanobis distance between the reconstructed projection vector and the text feature vector of each rock category to be identified is calculated. The Mahalanobis distance fully considers the covariance relationship of the feature variables and is more effective in removing noise interference in the feature space than traditional Euclidean distance and cosine similarity. It is also suitable for feature perturbations caused by factors such as lighting, occlusion, and weathering in complex outdoor scenes. Finally, the rock category with the smallest Mahalanobis distance is selected as the final classification result of the image to be classified and output. This achieves zero-shot classification capability that can complete high-precision identification of unseen rock categories without retraining.

[0048] It should be noted that, although the embodiments in this application are based on... Figure 1 Steps S101 to S104 will be described sequentially, but this does not mean that steps S101 and S104 must be performed in a strict order. The reason this embodiment follows this order is... Figure 1 The order in which steps S101 to S104 are described is provided to facilitate understanding of the technical solutions of the embodiments of this application by those skilled in the art. In other words, in the embodiments of this application, the order of steps S101 to S104 can be appropriately adjusted according to actual needs.

[0049] Figure 2 This is a diagram illustrating the architecture of a rock zero-sample classification model provided in this application embodiment.

[0050] exist Figure 2The paper demonstrates the overall model architecture based on CLIP-ViT-Base-Patch16, comprising three parts: image encoding branch, text encoding branch, and domain generalization processing branch. The image encoding branch sequentially shows the input rock image, image block embedding, multi-layer Transformer Encoder, dynamic enhancement module for rock micro-texture embedded in the outputs of the 5th, 6th, and 7th Transformer Encoders, and a cross-modal multi-granularity semantic adaptive alignment mechanism connected to the output of the 12th Transformer Encoder, ultimately outputting a fused and enhanced image feature vector. The text encoding branch shows the flow of text semantic description through the text encoder to output text feature vectors. It also demonstrates the connection relationships between the visual prototype library corresponding to the domain generalization visual prototype robust distillation method, the domain adversarial training unit, and the noise robust inference unit, as well as the overall logical link for cross-modal matching of image and text features.

[0051] Figure 3 The flowchart shows a rock microtexture dynamic enhancement module for a rock zero-sample classification method provided in this application embodiment.

[0052] exist Figure 3 The document demonstrates the internal processing flow of the dynamic enhancement module for rock microtexture. Using the Patch Embedding output from the TransformerEncoder as input, the process is divided into three branches: The first branch is the depthwise convolution kernel generation branch, showing the complete process of Unpatchify deserialization, parallel 1×1 convolution, global average pooling, and finally dynamically generating the depthwise convolution kernel; the second branch is the dual-branch channel attention mask generation branch, showing the processing chain where the channel attention branch and spatial attention branch extract weights respectively, fuse them, and then generate a normalized channel mask through a multilayer perceptron and activation function; the third branch is the texture feature reconstruction and enhancement branch, showing the complete process of using dynamic convolution kernels to perform depthwise separable convolution to extract texture features, combining the attention mask with the original features for weighted fusion, and finally serializing and restoring the output to PatchEmbedding.

[0053] Figure 4 A diagram illustrating the cross-modal, multi-granularity semantic adaptive alignment mechanism of a rock zero-sample classification method provided in this application embodiment.

[0054] exist Figure 4The document demonstrates the complete processing logic of a cross-modal multi-granularity semantic adaptive alignment mechanism, divided into three parts: multi-granularity feature extraction, text semantic gating, and adaptive fusion. It shows the process of extracting Class Tokens from layers 5, 8, and 12 of the image encoder, generating multi-granularity feature maps through Unpatchify and concatenation; it shows the process of obtaining mean, variance, and extreme value features from text features through statistical calculation, concatenating them to generate cross-modal global features, and then inputting them into a gating network to output adaptive weights; it also shows the complete flow of multi-granularity feature maps combined with adaptive weights, undergoing multi-level convolution and asymmetric fusion operations, ultimately generating an image feature vector containing macro and micro semantics.

[0055] Figure 5 A flowchart of a domain generalization visual prototype robust distillation method for a rock zero-sample classification method provided in this application embodiment.

[0056] exist Figure 5 The paper demonstrates the dual-flow training and inference phases of the domain generalization visual prototype robust distillation method. The training phase shows the process of constructing a visual prototype library by K-Means clustering of known class image features, as well as the complete training chain of calculating intra-class center loss, domain adversarial training branch composed of gradient inversion layer and domain classifier, and joint optimization of multiple losses. The inference phase shows the complete inference process of mapping the features of the image to be classified to the visual prototype feature manifold, calculating attention weights, generating reconstructed projection vectors, and completing similarity matching with text features through Mahalanobis distance.

[0057] Figure 6 A schematic diagram of a rock zero-sample classification device provided in this application embodiment includes: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform any of the above-mentioned rock zero-sample classification methods.

[0058] Some embodiments of this application provide a non-volatile computer storage medium for rock zero-sample classification, which stores computer-executable instructions capable of executing any of the above-described rock zero-sample classification methods.

[0059] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0060] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0061] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0063] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0064] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0065] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0066] Memory may include non-persistent storage in computer-readable media, random access memory (RAM), and non-volatile memory such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0067] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0069] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the technical principles of this application should fall within the protection scope of this application.

Claims

1. A rock zero-sample classification method, characterized in that, The method includes: Obtain rock images from multiple scenes and corresponding textual semantic descriptions for each rock image. Perform standardized preprocessing on each rock image to generate standardized training data. Based on the CLIP vision-language pre-training architecture, a zero-shot rock classification model is constructed. This model sequentially embeds a dynamic enhancement module for rock microtexture, a cross-modal multi-granularity semantic adaptive alignment mechanism, and a domain-generalized visual prototype robust distillation method. The dynamic enhancement module enhances rock microtexture feature extraction through dynamic convolution and bi-branch attention. The cross-modal multi-granularity semantic adaptive alignment mechanism achieves adaptive alignment between multi-granularity features of rock images and text semantics through multi-granularity feature fusion and text semantic guidance. The domain-generalized visual prototype robust distillation method compresses intra-class differences and eliminates scene offsets through visual prototype constraints and domain adversarial training. The rock zero-sample classification model is trained in two stages using the standardized training data to obtain the final rock zero-sample classification model. The rock image to be classified is input into the final rock zero-sample classification model. After feature extraction and cross-modal semantic matching calculation, the corresponding rock category classification result is output.

2. The method according to claim 1, characterized in that, Based on the CLIP vision-language pre-training architecture, a dynamic enhancement module for rock microtexture in a zero-shot rock classification model is constructed, specifically including: It is based on the CLIP-ViT-Base-Patch16 pre-trained model as the foundation, and retains the complete pre-trained parameters and network structure of the image encoder and text encoder; Rock microtexture dynamic enhancement modules are sequentially inserted at the output positions of the 5th, 6th, and 7th layers of the Transformer Encoder in the image encoder; The rock microtexture dynamic enhancement module is equipped with a depthwise convolution kernel generation unit. The input Patch Embedding is reverse-mapped into a two-dimensional feature map through the Unpatchify operation. After parallel convolutional layers and global average pooling, a depthwise convolution kernel corresponding to each feature position is generated. A dual-branch channel attention mask generation unit is set up to extract channel dimension weights and spatial location association weights respectively, and then merge them to generate a normalized channel attention mask. A texture feature reconstruction and enhancement unit is set up, and depth-separable convolution is performed through the depthwise convolution kernel to extract micro-texture features; Based on the microtexture features and the channel attention mask, the graphics encoder features and texture enhancement features are weighted and fused, and converted into a Patch Embedding of the corresponding dimension and output to the next layer of graphics encoder.

3. The method according to claim 1, characterized in that, Based on the CLIP vision-language pre-training architecture, a cross-modal, multi-granular semantic adaptive alignment mechanism is constructed for a rock zero-shot classification model, specifically including: A cross-modal, multi-granularity semantic adaptive alignment mechanism is inserted at the output of the 12th layer Transformer Encoder of the image encoder. The cross-modal multi-granularity semantic adaptive alignment mechanism includes extracting the ClassToken output from the 5th, 8th, and 12th layers of the image encoder, and concatenating it with the channel via Unpatchify to obtain multi-granularity feature maps corresponding to shallow edge contours, mid-layer surface textures, and deep overall structures. A text semantic-guided gating network is constructed, which generates cross-modal global features based on the mean, variance, and extreme value features of all categories of text features. The adaptive weights of the three granular features are then output through a weighted network. A multi-granularity feature adaptive asymmetric fusion unit is constructed, and the multi-granularity feature maps are weighted and fused step by step according to the adaptive weights to generate image feature vectors.

4. The method according to claim 1, characterized in that, Based on the CLIP vision-language pre-training architecture, a domain-generalized visual prototype robust distillation method for constructing a zero-shot rock classification model is presented, specifically including: A robust distillation method for embedding domain generalization visual prototypes; The domain generalized visual prototype robust distillation method extracts image feature vectors from rock images of each known category using the CLIP-ViT-Base-Patch16 pre-trained model, and performs K-Means clustering operations on all feature vectors of each rock category. The mean of the clustered vectors is used as the visual prototype of the rock category. The visual prototypes of all categories together constitute the visual prototype library. A domain classifier is connected after the image encoder, and a gradient inversion layer is introduced between the image encoder and the domain classifier to construct the zero-sample rock classification model.

5. The method according to claim 1, characterized in that, The rock zero-sample classification model is initially trained using the standardized training data, specifically including: All parameters of the image encoder and text encoder in the CLIP-ViT-Base-Patch16 base model are frozen, and the learnable parameters of the rock micro-texture dynamic enhancement module and the cross-modal multi-granularity semantic adaptive alignment mechanism are iteratively optimized to obtain a primary rock zero-sample classification model.

6. The method according to claim 5, characterized in that, The primary rock zero-sample classification model is jointly fine-tuned end-to-end using the standardized training data, specifically including: Unfreeze all parameters of the image encoder and text encoder in the CLIP-ViT-Base-Patch16 base model; Intra-class center loss and domain adversarial loss are introduced, which, together with CLIP native visual-language contrastive learning loss, constitute the total loss function to optimize the model parameters of the primary rock zero-shot classification model, thus obtaining the final rock zero-shot classification model.

7. The method according to claim 1, characterized in that, The process involves inputting the rock image to be classified into the final rock zero-sample classification model. After feature extraction and cross-modal semantic matching calculation, the model outputs the corresponding rock category classification result, specifically including: The rock image to be classified is input into the final rock zero-sample classification model to obtain the feature vector of the rock image to be classified. The feature vector is mapped to the feature manifold defined by the visual prototype library, the attention weights of all visual prototypes corresponding to the feature vector are calculated, and the reconstructed projection vector is obtained by weighted summation. Calculate the Mahalanobis distance between the reconstructed projection vector and the text feature vector of each rock category, and select the category with the smallest Mahalanobis distance as the rock classification result of the rock image to be classified.

8. The method according to claim 1, characterized in that, The standardization preprocessing of each rock image to generate standardized training data specifically includes: Each rock image is subjected to resolution unification and color space normalization processing; Each rock image after normalization is then subjected to automatic white balance and noise reduction processing in sequence; Histogram equalization is used to enhance the contrast of each denoised rock image. Each enhanced rock image is augmented to obtain standardized training data; the data augmentation includes random rotation, horizontal flipping, and brightness jitter.

9. A rock zero-sample classification device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform a rock zero-sample classification method according to any one of claims 1-8.

10. A rock zero-sample classification storage medium, storing computer-executable instructions, characterized in that, The computer-executable instructions are capable of executing the rock zero-sample classification method described in any one of claims 1-8.

Images