A 3D geological modeling method based on 3D-ViT model

Abstract

This application relates to the fields of geological modeling and artificial intelligence, and discloses a three-dimensional geological modeling method based on the 3D-ViT model. The method includes: acquiring three-dimensional geological models from multiple regions and performing mesh discretization and profile cutting; extracting three-dimensional sample blocks to construct a training dataset; constructing a deep learning model based on 3D Vision Transformer; inputting the training data into the model for iterative training to obtain a prediction model; acquiring sparse profile data of the area to be modeled and establishing a three-dimensional regular mesh; inputting the profile data into the prediction model, traversing the mesh cells to obtain predicted geological attribute values, and constructing a complete three-dimensional geological attribute model. The model in this application possesses geological location awareness capabilities, captures macroscopic geological structures, and considers both macroscopic and microscopic feature extraction, enabling it to quickly and accurately predict a complete three-dimensional geological attribute model under conditions of sparse profile data in new regions.

Description

Technical Field

[0001] This application belongs to the field of geological modeling and artificial intelligence, specifically involving a three-dimensional geological modeling method based on the 3D-ViT model. Background Technology

[0002] Three-dimensional geological modeling is a crucial foundation for mineral resource exploration, underground engineering construction, and geological research. By establishing a three-dimensional geological model of the study area, the underground geological structure can be visually reflected, providing important data for resource evaluation, deposit prediction, and engineering design. In practical applications, due to limitations in exploration costs and technical conditions, only limited profile data is often obtained, while large areas between profiles lack detailed geological attribute information. How to accurately infer a complete three-dimensional geological model from this sparse profile data has always been a core technical challenge in the field of geological modeling.

[0003] Traditional 3D modeling methods typically rely on interpolation algorithms or implicit modeling methods. When dealing with complex geological structures, these methods often require extensive human experience and expert knowledge to interpret and model borehole data, profile data, and geological maps. While these methods can reflect the basic characteristics of geological structures to some extent, the modeling process is usually complex and cumbersome, and they struggle to effectively represent the intricate spatial relationships between geological bodies. This is especially true in areas with irregular geological interfaces and varied geological body morphologies, where the reliability of the modeling results is difficult to guarantee. Furthermore, traditional methods heavily depend on the completeness and accuracy of the data; when the data is sparse or incomplete, the modeling effectiveness significantly decreases.

[0004] In recent years, the Vision Transformer model has made groundbreaking progress in the field of computer vision. With its self-attention mechanism, the Vision Transformer can establish long-range dependencies between any positions in an input sequence, demonstrating great potential in capturing global contextual information. This characteristic provides a novel technical approach for 3D spatial structure modeling. However, the standard Vision Transformer model still has significant shortcomings when directly applied to geological modeling: First, the standard ViT lacks a priori encoding mechanism for geological locations, failing to effectively utilize location information unique to the geological domain, such as profile locations and spatial distances; second, in sparse data scenarios, where only profile locations have accurate labels while other regional information is missing, directly training the standard ViT model faces significant challenges, making it difficult for the model to learn effective geological feature representations, resulting in insufficient prediction accuracy.

[0005] To address the aforementioned issues, this invention proposes a three-dimensional geological modeling method based on the 3D-ViT model. By constructing a deep learning model architecture specifically designed for the characteristics of geological data, it integrates geological profile information and spatial structure information, and uses existing three-dimensional geological models from multiple regions as training data to learn the general feature representation of geological structures. This enables rapid and accurate prediction of complete three-dimensional geological attribute models in new areas with only sparse profile data, providing an efficient and reliable technical solution for predicting the spatial distribution of geological bodies under complex geological conditions. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a three-dimensional geological modeling method based on the 3D-ViT model. It utilizes existing three-dimensional geological models from multiple regions as training samples and learns the general feature representation of geological structures through a deep learning framework. This enables accurate prediction of three-dimensional geological attributes in regions with only sparse profile data, effectively overcoming the technical defect of traditional methods where accuracy drops significantly when processing large-scale sparse data.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A three-dimensional geological modeling method based on the 3D-ViT model includes the following steps: Step S1: Obtain three-dimensional geological models that have been constructed in multiple regions as training sample data, perform three-dimensional mesh discretization processing on the three-dimensional geological models, and cut the model into several two-dimensional profiles at fixed intervals along a certain coordinate axis, wherein all mesh units on each profile have complete geological attribute values. Step S2: Extract 3D sample blocks from the profile data of the constructed model, divide them into training set and validation set according to a preset ratio, and construct the model training dataset; Step S3: Construct a deep learning model based on 3D Vision Transformer. This model consists of a position conditional encoder, a condition-aware 3D Patch embedding module, a 3D position encoding module, a local-global attention module, a Transformer encoder module, and a dual-task classification head. Step S4: Input the training dataset into the 3D Vision Transformer model for iterative training to obtain a trained geological attribute prediction model; Step S5: Obtain sparse profile data of the area to be modeled, establish a three-dimensional regular grid in the same way as the training data, assign spatial coordinates to each grid cell, and initially set the geological attribute values ​​to empty; Step S6: Input the profile data of the area to be predicted into the trained geological attribute prediction model, traverse all grid cells, obtain the geological attribute prediction value of each cell, and then construct a complete three-dimensional geological attribute model.

[0008] Furthermore, step S1 specifically includes: The 3D mesh discretization process divides each 3D geological model into regular mesh cells of equal size, assigns spatial coordinates to each mesh cell, and assigns corresponding geological attribute values ​​to the mesh cells on the profile.

[0009] The training sample set is constructed by extracting a fixed-size 3D neighborhood block centered on each grid cell. For voxels within the window, only the attribute values ​​located at the cross-sectional position are retained, and the other positions are marked as invalid values. The attribute value of the central cell of the 3D neighborhood block is used as the label of the sample, and a corresponding condition vector is generated for each sample.

[0010] The sample set is constructed using a hierarchical sampling strategy, which divides the grid cells into multiple distance groups based on the distance from the nearest profile, and samples are taken evenly from each group to ensure that the model learns geological features at different distances.

[0011] Furthermore, step S3 specifically includes: The position condition encoder is used to encode position information into a condition vector, which includes whether it is a profile, the distance to the nearest profile, normalized 3D coordinates, interpolation weights, and profile index.

[0012] The positional condition encoder consists of three core components: a main encoder, a profile attention module, and a continuity perception module. The main encoder uses a three-layer fully connected network structure to extract basic information about the positional conditions and outputs the basic positional code. The profile attention module uses a two-layer fully connected network structure to generate profile attention weights based on the conditional vector, which are used to adjust the intensity of features at different positions, enabling the model to distinguish between profile and non-profile positions. The continuity perception module uses a three-layer fully connected network structure to output continuity features, which are used to learn continuity patterns between profiles and enhance the model's ability to utilize information from adjacent profiles at non-profile positions.

[0013] The final enhanced location encoding is obtained by multiplying the base location encoding element-wise with the profile attention weights and then adding the continuous features.

[0014] Furthermore, step S3 specifically includes: The condition-aware 3D Patch embedding module uses 3D convolution to segment the input 3D neighborhood block into multiple 3D blocks and map them into embedding vectors. The specific implementation includes two parts: a basic projection layer and a context feature extraction layer.

[0015] The basic projection layer uses a 3D convolution with a kernel size of one and a stride of one to map the patch into an embedding vector. The context feature extraction layer uses two consecutive 3D convolutions with a kernel size of three and padding of one. Each convolution is followed by a GELU activation function to extract local context features. The basic features and context features are added together to obtain the comprehensive features, which are then flattened and transposed to obtain the sequence features.

[0016] When embedding voxel blocks, the conditional vector is expanded to the sequence length by the position conditional encoder and then fused with the Patch embedding features through a cross-attention mechanism. At the same time, a gating mechanism is used to achieve deep interaction between features and position conditions, enabling the model to perceive spatial location and profile information and guiding the model to pay more attention to key geological locations.

[0017] Furthermore, step S3 specifically includes: The 3D position encoding module initializes independent learning position encoding matrices for the three dimensions of depth, height, and width. The position encoding matrices in the three directions correspond to the spatial position information in different directions in 3D space.

[0018] When adding position encoding, the three-dimensional mesh position index is first calculated, and the encodings of the three dimensions of depth, height and width are added and fused to generate a three-dimensional absolute position encoding containing multi-scale information. The fused position encoding is added to the sequence features output by the condition-aware 3D Patch embedding module to realize the injection of three-dimensional absolute position information. Finally, a classification token is added to aggregate global information.

[0019] Furthermore, step S3 specifically includes: The Transformer encoder module consists of multiple enhanced Transformer encoding blocks stacked together, each encoding block containing a first processing branch and a second processing branch.

[0020] The first processing branch includes, in sequence, a layer normalization unit, a conditional injection unit, a local-global attention module, and a drop path unit, with each unit connected via residual connections; the second processing branch includes, in sequence, a second layer normalization unit, a multilayer perceptron, and a second drop path unit, with each unit also connected via residual connections.

[0021] A conditional injection mechanism is set in some coding blocks. The conditional injection unit obtains the projected conditional features from the conditional vector through the projection layer, expands it to the same shape as the feature sequence, and concatenates it with the normalized features. The gate weights are generated by the Sigmoid activation function. The gate weights are multiplied element-wise with the projected conditional features and then added to the normalized features to achieve dynamic fusion of conditional information.

[0022] Furthermore, step S3 specifically includes: The local-global attention module is integrated into each coding block of the Transformer encoder module. This module includes two parallel branches: a global multi-head self-attention sublayer and a local attention sublayer.

[0023] The global multi-head self-attention sublayer adopts the standard multi-head self-attention mechanism to calculate the attention relationship between all tokens and establish long-range dependencies between arbitrary voxels. It is suitable for capturing macroscopic geological structures such as fault extension and ore body strike.

[0024] The local attention sublayer extracts tokens other than classification tokens, reshapes the features into a three-dimensional structure, and uses three-dimensional convolution to perform feature aggregation within the local neighborhood of the token sequence, focusing on the local detailed features of adjacent voxels. It is suitable for characterizing microscopic geological phenomena such as alteration zoning and lithological boundaries.

[0025] The global attention output and the local attention output are weighted and added together in a ratio of 0.5:0.5 to obtain the fused attention output, which is then output through linear projection and a dropout layer.

[0026] This dual attention mechanism allows the model to take into account both long-range dependencies and local details, making it more consistent with the multi-scale characteristics of geological bodies and improving the accuracy of predictions.

[0027] Furthermore, step S3 specifically includes: The dual-task classification head consists of two classifiers: a main classification head and an auxiliary classification head.

[0028] The main classification head is a distance-aware classification head. It dynamically fuses the outputs of multiple sub-classifiers designed for different distance ranges based on the distance from the input sample to the profile. For areas with closer distances, the prediction results of the close-range sub-classifier are used as the main factor. For areas with farther distances, the prediction results of the far-range sub-classifier are used as the main factor. For intermediate areas, the outputs of each sub-classifier are weighted and fused to obtain the final geological attribute category prediction.

[0029] The auxiliary classification head is a binary classifier used to predict whether the current input sample unit block is located on the profile line. The auxiliary task guides the model learning and enhances the model's ability to distinguish features between profile and non-profile locations.

[0030] Furthermore, step S4 specifically includes: During model training, a multi-task weighted loss function is used, and the total loss is obtained by weighted summation of four losses: category loss, location loss, distance loss, and continuity loss.

[0031] The category loss uses the cross-entropy loss function, and the category weights are calculated based on the reciprocal of the number of samples in each category in the training set for balancing. The location loss uses the binary cross-entropy loss function. The distance loss uses the mean squared error loss function, which is calculated only for the profile location. The continuity loss uses the mean squared error loss function, and the target value is the continuity weight calculated based on the consistency of adjacent profiles.

[0032] During training, the classification accuracy of the validation set is calculated after each training round. The loss values ​​of the training set and the validation set are recorded and plotted as a curve to analyze the training convergence state and adjust the hyperparameter settings.

[0033] In summary, this application includes at least one of the following beneficial technical effects: This invention is the first to apply the 3D vision Transformer architecture to the field of 3D geological modeling. By utilizing the Transformer's self-attention mechanism, it effectively captures long-range dependencies in 3D space, overcoming the limitations of traditional convolutional neural networks with their limited receptive fields. This allows for better modeling of complex geological structural spatial relationships. The global attention mechanism can establish long-range dependencies between arbitrary voxels, making it suitable for capturing macroscopic geological structures such as fault extensions and ore body strikes.

[0034] This invention features a location-condition encoder specifically designed for the characteristics of geological data. It encodes prior information such as profile attributes, distances, and coordinates into conditional vectors, enabling the model to possess geological location awareness and effectively solving the technical problem of inaccurate prediction in sparse data scenarios. The location-condition encoder works collaboratively through three core components: a main encoder, a profile attention module, and a continuity awareness module. It can learn and express hierarchical features of location conditions. Simultaneously, the condition-aware 3DPatch embedding module incorporates spatial coordinates and boundary markers as conditional information into the model, guiding the model to focus more on key geological locations. Through cross-attention and gating mechanisms, it achieves deep interaction between features and location conditions, significantly improving the model's feature representation capabilities.

[0035] This invention separates the training and application phases. The training phase utilizes existing 3D geological models from multiple regions as training data, enabling the model to learn common geological structural features across regions and geological types, thus possessing excellent cross-regional generalization ability. The application phase only requires sparse profile data from new regions to quickly generate a complete 3D geological model. Simultaneously, the construction of the training sample set employs a hierarchical sampling strategy, dividing the grid cells into different distance groups based on their distance from the nearest profile. Equal sampling is then performed from each group to ensure the model learns features at different distances, further improving the model's adaptability and prediction accuracy. Attached Figure Description

[0036] Figure 1This is a schematic diagram of the complete technical solution architecture of the three-dimensional geological modeling method based on the 3D-ViT model proposed in this invention; Figure 2 This is a block diagram of the overall structure of the 3D-ViT model of the present invention; Figure 3 This is a schematic diagram of the Transformer encoder of the present invention; Figure 4 This is a graph showing the model training loss and accuracy in an embodiment of the present invention. Figure 5 This is a schematic diagram of the prediction results of the three-dimensional geological model in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The present invention will be further described in detail below with reference to specific embodiments.

[0038] This invention provides a three-dimensional geological modeling method based on the 3D-ViT model. This method constructs a three-dimensional visual Transformer deep learning architecture specifically designed for the characteristics of geological data, learns the distribution law of geological attributes from sparse profile data, and finally realizes the prediction of geological attributes in three-dimensional space.

[0039] like Figure 1 As shown, the three-dimensional geological modeling method of this invention includes two main parts: a model training stage and an application stage. The model training stage corresponds to steps S1 to S4, and the application stage corresponds to steps S5 to S6. In the model training stage, three-dimensional geological models already constructed in multiple regions are used as training sample data. After being discretized by a three-dimensional mesh, the data is cut into several profile data to construct a training dataset, which is then input into the 3D-ViT model for iterative training. In the application stage, sparse profile data is used as input, and a complete three-dimensional geological attribute model is generated by predicting using the trained model.

[0040] In one specific embodiment of the present invention, a detailed explanation is provided using the modeling of five types of lithological or geological attribute units in a certain study area as an example. This embodiment fully demonstrates the entire technical implementation process from data preparation to model training and application prediction.

[0041] Step S1: 3D geological model preprocessing and profile cutting The existing three-dimensional geological models of the study area were collected as training sample data. For example... Figure 2As shown, the three-dimensional geological model is cut into several two-dimensional profile data at fixed intervals along a certain coordinate axis. Each profile has continuous geological attribute records, while the areas between the profiles have no geological attribute data. This processing method simulates the situation in real exploration scenarios where only the profiles have detailed data.

[0042] After the profile was cut, a three-dimensional regular grid covering the entire area was created based on the profile. The grid size was set to 100 meters × 100 meters × 100 meters, with the profile distributed along the X-direction and a spacing of 10 grid cells between profiles. The profile data was projected onto the three-dimensional grid, and each grid cell was assigned an initial attribute value. Accurate geological attribute values ​​were assigned to profile locations, while non-profile locations were marked with a negative one to indicate unknown data. Finally, a 100 × 100 × 100 three-dimensional attribute array was obtained as input data for subsequent processing.

[0043] The 3D mesh discretization process divides each 3D geological model into regular mesh cells of equal size, assigning spatial coordinates (X, Y, and Z coordinates) to each mesh cell, and simultaneously assigning corresponding geological attribute values ​​to the mesh cells on the profile. This discretization process establishes a meshed representation of geological attributes in 3D space, providing a standardized data format for subsequent deep learning model processing.

[0044] Step S2: Construction and partitioning of training sample dataset Based on the existing 3D geological model of the study area, each grid cell is assigned a geological attribute classification label. Using the grid cell with the classification label as the center, a fixed-size 3D neighborhood block is extracted as a training sample.

[0045] The specific extraction parameters are as follows: the side length of the 3D neighborhood block is set to 7, that is, the window radius is equal to 3. To ensure that the window does not exceed the grid boundary, only grid cells with center coordinates in the range of 3 to 96, i.e., at an appropriate distance within the boundary, are considered as the sample center points.

[0046] For each extracted 3D window, a sparse label window is generated. Specifically, for voxels within the window, only the attribute values ​​located at the cross-sectional positions are retained, and the remaining positions are marked as invalid values.

[0047] The central cell attribute value of the 3D neighborhood block serves as the label for the sample, and a corresponding condition vector is generated for each sample. The condition vector contains information in seven dimensions: whether it is a profile location, the normalized distance to the nearest profile, the normalized X coordinate, the normalized Y coordinate, the normalized Z coordinate, the interpolation weight, and the normalized profile index.

[0048] Considering the uneven distribution of the original labeled voxels, this embodiment of the invention employs a stratified sampling strategy. Specifically: All possible center points were divided into multiple distance groups based on their distance to the nearest profile. Within each distance group, samples were taken from each category proportionally, resulting in approximately 8,000 samples per category. The final training sample size was 40,000, with 8,000 samples per category, and profile location samples comprising approximately 25%. This hierarchical sampling strategy ensured that the model could learn geological features under different distance conditions, preventing the model from overemphasizing samples within a specific distance range while ignoring samples within other distance ranges.

[0049] The sample set was randomly partitioned into training and validation datasets in an 8:2 ratio to ensure a consistent proportion of each class in both sets. This partitioning method guarantees a sufficient number of samples for model training while retaining independent validation data for evaluating model performance and selecting the optimal model.

[0050] Step S3: Construction of 3D-ViT Deep Learning Model A deep learning model based on the 3D vision Transformer, namely the 3D-ViT model, is constructed. The input of this model is a 7×7×7 sparse window and the corresponding conditional vector. For example... Figure 2 As shown, the core architecture of the 3D-ViT model includes a position conditional encoder, a condition-aware 3D Patch embedding module, a 3D learnable position encoding module, a local-global attention module, a Transformer encoder module, and a dual-task classification head.

[0051] (1) Position condition encoder module The position condition encoder is used to encode position information into condition vectors. The encoder consists of three core components: the main encoder, the profile attention module, and the continuity perception module.

[0052] The main encoder is implemented using a three-layer fully connected network structure. It receives a seven-dimensional conditional vector as input and outputs basic positional codes to extract fundamental information about the positional conditions. The profile attention module is implemented using a two-layer fully connected network structure. It generates profile attention weights based on the conditional vector to adjust the intensity of features at different locations, enabling the model to distinguish between profile and non-profile locations. The continuity perception module is implemented using a three-layer fully connected network structure. It outputs continuity features to learn continuity patterns between profiles, enhancing the model's ability to utilize information from adjacent profiles at non-profile locations.

[0053] The final enhanced positional encoding is obtained by multiplying the basic positional encoding element-wise with the profile attention weights and adding continuous features. This processing method realizes a hierarchical expression of positional conditions.

[0054] (2) Condition-aware 3D Patch Embedding Module The condition-aware 3D Patch embedding module uses 3D convolution to segment the input 3D neighborhood block into multiple 3D blocks and map them into embedding vectors. The specific implementation includes two parts: a basic projection layer and a context feature extraction layer.

[0055] The base projection layer uses a 3D convolution with a kernel size of one and a stride of one to map the patch into an embedding vector. This process converts the input 3D window into a feature sequence. The context feature extraction layer uses two consecutive 3D convolutions, each with a kernel size of three and padding of one. Each convolution is followed by a GELU activation function to extract local context features. The base features and context features are then added together to obtain the combined features, which are then flattened into two dimensions and transposed to obtain the sequence features.

[0056] When embedding voxel blocks, the conditional vector is expanded to the sequence length by the position conditional encoder and then fused with the patch embedding features through a cross attention mechanism. At the same time, a gating mechanism is used to achieve deep interaction between features and position conditions. This design enables the model to perceive spatial location and profile information, guiding the model to pay more attention to key geological locations.

[0057] (3) Three-dimensional learnable position coding module The 3D learnable position encoding module initializes independent learning position encoding matrices for the three dimensions of depth, height and width. The position encoding matrices in the three directions correspond to the spatial position information in different directions in 3D space.

[0058] When adding position encoding, the three-dimensional mesh position index is first calculated, and the encodings of the three dimensions of depth, height and width are added and fused to generate a three-dimensional absolute position encoding containing multi-scale information. The fused position encoding is added to the sequence features output by the condition-aware 3D Patch embedding module to realize the injection of three-dimensional absolute position information. Finally, a classification token is added to aggregate global information.

[0059] (4) Transformer encoder module The Transformer encoder module consists of multiple stacked enhanced Transformer encoder blocks and is the core feature extraction part of the 3D-ViT model.

[0060] like Figure 3As shown, each coding block contains two parallel branch structures: a first processing branch and a second processing branch. The first processing branch sequentially includes a layer normalization unit, a conditional injection unit, a multi-head self-attention mechanism, and a DropPath unit, with each unit connected via residual connections. The second processing branch sequentially includes a second layer normalization unit, a multilayer perceptron, and a second DropPath unit, with each unit also connected via residual connections.

[0061] In this embodiment of the invention, the Transformer encoder module stacks three encoding blocks. The processing flow for each encoding block is as follows: First, a first layer of normalization is performed. Then, conditional injection is performed based on the coding block index. The conditional injection unit processes the following: the conditional vector is projected through a projection layer to obtain projected conditional features, which are then expanded to the same shape as the feature sequence and concatenated with the normalized features. A sigmoid activation function is used to generate gating weights, which are then multiplied element-wise with the projected conditional features and added to the normalized features to achieve dynamic fusion of conditional information. After conditional injection, the result is input into the local-global attention module to obtain the attention output, which is then passed through a DropPath unit and added to the original input to achieve residual connection. Next, a second layer of normalization is performed, input into the multilayer perceptron to obtain the MLP output, and added to the attention residual result. After all coding blocks are processed, a final layer normalization is performed to obtain the encoder output sequence.

[0062] (5) Local-Global Attention Module The local-global attention module is integrated into each coding block of the Transformer encoder module, including two parallel branches: a global multi-head self-attention sub-layer and a local attention sub-layer.

[0063] The global multi-head self-attention sublayer employs a standard multi-head self-attention mechanism to calculate the attention relationships between all tokens and establish long-range dependencies between arbitrary voxels. This mechanism is suitable for capturing macroscopic geological structures such as fault extension and orebody strike. The local attention sublayer extracts tokens other than classification tokens, reshapes the features into a 3D structure, and uses 3D convolution to aggregate features within the local neighborhood of the token sequence, focusing on the local detailed features of adjacent voxels. This mechanism is suitable for characterizing microscopic geological phenomena such as alteration zoning and lithological boundaries. The global attention output and the local attention output are weighted and added at a ratio of 0.5:0.5 to obtain a fused attention output, which is then output through linear projection and a dropout layer. This dual attention mechanism allows the model to consider both long-range dependencies and local details, better reflecting the multi-scale characteristics of geological bodies and significantly improving the accuracy of predictions.

[0064] (6) Dual-task classification head The dual-task classification head consists of two classifiers: a main classification head and an auxiliary classification head.

[0065] The main classification head is a distance-aware classification head that dynamically fuses the outputs of multiple sub-classifiers designed for different distance ranges based on the distance from the input sample to the profile. For closer regions, the prediction results from the near-distance sub-classifier are used as the primary method; for farther regions, the prediction results from the far-distance sub-classifier are used as the primary method; and for intermediate regions, the outputs of each sub-classifier are weighted and fused to obtain the final geological attribute category prediction. The auxiliary classification head is a binary classifier used to predict whether the current input sample unit block is located on the profile line. Through auxiliary tasks, the model is guided to learn, enhancing its ability to distinguish features between profile locations and non-profile locations.

[0066] Step S4: Model Training and Validation The training dataset constructed in step S2 is input into the 3D-ViT model for model training. This embodiment of the invention uses the PyTorch deep learning framework to implement the model. The specific training configuration parameters are as follows: the optimizer is AdamW, the initial learning rate is set to 0.001, and the weight decay is set to 0.0001; the learning rate scheduling uses a cosine annealing strategy, the maximum number of iterations is set to 50; the batch size is set to 64; and the number of training rounds is set to 50.

[0067] The loss function adopts a multi-task weighted loss function. The total loss is obtained by weighted summation of four losses: category loss, location loss, distance loss, and continuity loss. The calculation formula is as follows: .

[0068] The class loss employs the cross-entropy loss function, and class weights are calculated based on the reciprocal of the number of samples in each class in the training set for balancing. This approach avoids the impact of class imbalance on model training. The location loss uses a binary cross-entropy loss function to assist the classifier in predicting whether a grid cell is located within a profile. The distance loss uses the mean squared error loss function, calculated only for profile locations, to learn the continuity features of distance. The continuity loss also uses the mean squared error loss function, with the objective value being a pre-calculated continuity weight based on the consistency of adjacent profiles. This design enhances the model's ability to model the spatial continuity of geological bodies.

[0069] During training, the classification accuracy of the validation set is calculated after each training round. The loss values ​​of the training set and the validation set are recorded and plotted as curves to analyze the training convergence state and adjust the hyperparameter settings.

[0070] like Figure 4As shown, the accuracy subplot illustrates the trends of training and test accuracy with each training epoch, while the loss subplot shows the convergence of training and test losses with each training epoch. The validation set accuracy gradually increases from approximately 20% in the initial random state, reaching a stable state around the 35th training epoch, indicating that the model has fully learned the geological features and patterns in the training data. The model with the highest validation set accuracy is saved as the optimal model to a specified file, completing the model training process.

[0071] Step S5: Data preparation for the area to be modeled Sparse profile data of the region to be modeled is obtained. This region is a new area without a complete 3D geological model, possessing only partial profile data. Profile data for this region is collected, with continuous geological attribute records on each profile. A 3D regular mesh is constructed based on the profile data, following the same method as the training data. Spatial coordinates are assigned to each mesh cell, and the geological attribute values ​​are initially empty. This processing method ensures that the input data format is completely consistent with the input format used during model training, providing standardized data input for subsequent model predictions.

[0072] Step S6: 3D Geological Attribute Prediction and Model Building Load the best-trained model from step S4 and predict each grid cell in the study area. For each grid cell, extract a 7×7×7 window centered on it as the model input. Input the sparse window and conditional vector into the trained 3D-ViT model. The model outputs the five-class probability distribution of that cell, and the class corresponding to the highest probability is taken as the predicted attribute value. After traversing all grid cells, a complete three-dimensional geological attribute model is obtained.

[0073] like Figure 5 As shown in the diagram, the three-dimensional geological model prediction results illustrate the complete implementation process of the method. The left side displays sparse profile data as input, and the right side displays the complete three-dimensional geological model as output, intuitively presenting the conversion effect from sparse profile to continuous three-dimensional model. The modeling results can accurately reflect the geological structural characteristics of the study area, including macroscopic geological structures such as fault extension and ore body strike, as well as microscopic geological phenomena such as alteration zoning and lithological boundaries.

[0074] The three-dimensional geological modeling method of this invention, through the above technical steps, successfully applies the 3D-ViT model to the field of three-dimensional geological modeling. It fully utilizes the global modeling capabilities of the Transformer and optimization strategies designed specifically for the characteristics of geological data, effectively improving the accuracy and reliability of geological attribute prediction. This method is the first to apply the three-dimensional vision Transformer architecture to the field of three-dimensional geological modeling. Through the Transformer's self-attention mechanism, it effectively captures long-distance dependencies in three-dimensional space, overcoming the technical limitation of the limited receptive field in traditional convolutional neural networks, and is able to better model complex spatial relationships of geological structures.

[0075] In some optional embodiments of the present invention, the mesh size of the 3D mesh can be adjusted according to actual application requirements. Larger mesh sizes are suitable for modeling large areas, while smaller mesh sizes are suitable for fine-scale modeling. The profile spacing can be adjusted according to the density of geological data acquisition. Sparse profiles are suitable for the preliminary modeling stage, while dense profiles are suitable for the fine-scale modeling stage. The number of Transformer encoding blocks can be selected according to the balance between model complexity and computational resources. More encoding blocks can extract deeper feature representations, but will increase computational overhead. The fusion ratio of the attention mechanism can be adjusted according to the specific application scenario. When more attention needs to be paid to macroscopic geological structures, the weight of global attention can be increased; when more attention needs to be paid to microscopic geological phenomena, the weight of local attention can be increased.

[0076] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0077] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A three-dimensional geological modeling method based on a three-dimensional visual Transformer model, comprising the following steps: Obtain existing 3D geological models from multiple regions as training sample data; The three-dimensional geological model is discretized into a three-dimensional mesh, and the model is cut into several two-dimensional profiles at fixed intervals along a certain coordinate axis, wherein all mesh cells on each profile have complete geological attribute values. Extract 3D sample blocks from the profile data of the constructed model, divide them into training set and validation set according to a preset ratio, and construct the model training dataset; A deep learning model based on 3D vision Transformer is constructed. This model consists of a position conditional encoder, a condition-aware 3D Patch embedding module, a 3D position encoding module, a local-global attention module, a Transformer encoder module, and a dual-task classification head. The training dataset is input into the 3D vision Transformer model for iterative training to obtain a trained geological attribute prediction model. Acquire sparse profile data of the area to be modeled, establish a three-dimensional regular grid in the same way as the training data, assign spatial coordinates to each grid cell, and initially set the geological attribute values ​​to empty; The profile data of the area to be predicted is input into the trained geological attribute prediction model. All grid cells are traversed to obtain the geological attribute prediction value of each cell, thereby constructing a complete three-dimensional geological attribute model.

2. The three-dimensional geological modeling method as described in claim 1, characterized in that, The three-dimensional mesh discretization process divides each three-dimensional geological model into regular mesh units of equal size, assigns spatial coordinates to each mesh unit, and assigns corresponding geological attribute values ​​to the mesh units on the profile.

3. The three-dimensional geological modeling method as described in claim 2, characterized in that, The training sample set is constructed by extracting a fixed-size 3D neighborhood block centered on each grid cell. For voxels within the window, only the attribute values ​​located at the cross-sectional position are retained, and the other positions are marked as invalid values. The attribute value of the central cell of the 3D neighborhood block is used as the label of the sample, and a corresponding condition vector is generated for each sample.

4. The three-dimensional geological modeling method as described in claim 3, characterized in that, The sample set is constructed using a hierarchical sampling strategy, which divides the grid cells into multiple distance groups based on the distance from the nearest profile, and samples are taken evenly from each group.

5. The three-dimensional geological modeling method as described in claim 1, characterized in that, The position condition encoder is used to encode position information into a condition vector. The position information includes whether it is a profile, the distance to the nearest profile, normalized three-dimensional coordinates, interpolation weights, and profile index.

6. The three-dimensional geological modeling method as described in claim 5, characterized in that, The position condition encoder consists of three core components: a main encoder, a profile attention module, and a continuous perception module. The main encoder uses a three-layer fully connected network structure to extract basic information about the position conditions and outputs a basic position code. The profile attention module adopts a two-layer fully connected network structure and generates profile attention weights based on the condition vector. The continuity sensing module adopts a three-layer fully connected network structure and outputs continuity features; The final enhanced location encoding is obtained by multiplying the base location encoding element-wise with the profile attention weights and then adding the continuous features.

7. The three-dimensional geological modeling method as described in claim 1, characterized in that, The condition-aware 3D Patch embedding module uses 3D convolution to segment the input 3D neighborhood block into multiple 3D blocks and map them into embedding vectors. This module includes a basic projection layer and a context feature extraction layer.

8. The three-dimensional geological modeling method as described in claim 7, characterized in that, The basic projection layer uses a 3D convolution with a kernel size of one and a stride of one to map the patch into an embedding vector; the context feature extraction layer uses two consecutive 3D convolutions with a kernel size of three and padding of one, followed by a GELU activation function after each convolution; the conditional vector is expanded to the sequence length by a positional conditional encoder and then fused with the patch embedding features through a cross-attention mechanism, while a gating mechanism is used to achieve deep interaction between features and positional conditions.

9. The three-dimensional geological modeling method as described in claim 1, characterized in that, The 3D position encoding module initializes independent learning position encoding matrices for the depth, height, and width dimensions, respectively. The position encoding matrices in the three directions correspond to spatial position information in different directions in 3D space. When adding position encoding, the 3D mesh position index is first calculated, and the encodings in the depth, height, and width directions are added and fused to generate a 3D absolute position encoding containing multi-scale information. The fused position encoding is added to the sequence features output by the condition-aware 3D Patch embedding module, and finally a classification token is added to aggregate global information.

10. The three-dimensional geological modeling method as described in claim 1, characterized in that, The Transformer encoder module is composed of multiple enhanced Transformer coding blocks stacked together. Each coding block contains two branch structures: a first processing branch and a second processing branch. The first processing branch includes, in sequence, a layer normalization unit, a conditional injection unit, a local-global attention module, and a drop path unit. The units are connected through residual connections. The second processing branch includes, in sequence, a second layer normalization unit, a multilayer perceptron, and a second drop path unit. The units are also connected through residual connections.

Images