An improved modeling method of UCaps deep learning framework

Abstract

This application belongs to the interdisciplinary field of computer vision and earth science, and specifically discloses an improved modeling method for the UCAPs deep learning framework, including: acquiring raw borehole data, performing coordinate system transformation and rotation correction, dividing voxel meshes and interpolating to generate input tensors; extracting preliminary geometric features using 3D convolutional layers; converting scalar neurons into vector capsules through 3D convolution to encode the existence probability and spatial pose of geological entities; applying capsule convolution modules for clustering, introducing a multi-head self-attention mechanism in the bottleneck layer to establish voxel associations across the entire image, and using graph Laplacian sampling to preserve the topological skeleton of geological bodies; restoring resolution through upsampling and skip connections, and outputting the conditional probability distribution of geological categories; and iteratively updating network parameters based on this distribution using a joint objective function to achieve 3D geological attribute field reconstruction. This application can achieve high-precision, structurally consistent 3D reconstruction under constraints of sparse data and complex geological structures.

Description

Technical Field

[0001] This application belongs to the interdisciplinary field of computer vision and earth science, and more specifically, relates to a modeling method for an improved UCAPs deep learning framework. Background Technology

[0002] Three-dimensional geological modeling is a core technology for revealing the structure, attribute distribution, and evolution of underground space, playing an irreplaceable role in mineral resource exploration, hydrogeological analysis, groundwater migration simulation, and urban stratigraphic planning. Accurate three-dimensional geological models can not only assist in scientific decision-making but also significantly reduce engineering risks and exploration costs. However, in practical applications, due to the invisibility of underground information, three-dimensional geological modeling faces multiple severe challenges. First, geological observation data is often extremely scarce and unevenly distributed, resulting in fragmented, sparse, and irregular underground information. How to reconstruct a continuous, high-precision three-dimensional geological attribute field from these limited observation points is the core challenge of geological modeling.

[0003] Currently, existing 3D geological modeling technologies are mainly divided into the following two categories, but both have obvious limitations: Traditional geostatistical interpolation methods, such as Kriging, Inverse Distance Weighted (IDW), and Nearest Neighbor (NN) interpolation, are widely used in geological modeling. However, these methods typically rely on stationarity or linear assumptions, making it difficult to capture the complex nonlinear spatial correlations implicit in stratigraphic sequences. Furthermore, these methods lack effective constraints on the topological structure of geological bodies, and when faced with complex geological structures such as faults, folds, or discrete lenses, the models are prone to boundary blurring and physically unreasonable staircase effects.

[0004] In recent years, deep learning methods based on convolutional neural networks (CNNs), represented by 3D U-Net, have brought new opportunities to 3D modeling, with their multi-scale feature extraction capabilities enabling the learning of complex spatial patterns. However, conventional 3D CNNs still have inherent technical bottlenecks, including: limited receptive field (convolutional operations are limited to fixed local receptive fields, making it difficult to capture geological information with long-range spatial dependence features, such as long-distance fault extensions and sedimentary facies changes); lack of spatial pose representation ability (scalar neurons in CNNs can only record the presence or absence of features, lacking the ability to explicitly express the spatial pose of geological entities); and loss of structural information (traditional pooling operations often rely on simple numerical filtering during downsampling, easily leading to the loss of topological skeleton information of geological bodies, making it difficult to maintain the physical consistency of strata).

[0005] Therefore, how to achieve high-precision and structurally consistent 3D reconstruction under the constraints of sparse data and complex geological structures is an urgent problem to be solved. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this application is to provide an improved modeling method for the UCAPs deep learning framework, which can achieve high-precision and structurally consistent 3D reconstruction under constraints of sparse data and complex geological structures.

[0007] To achieve the above objectives, in a first aspect, this application provides an improved modeling method for the UCAPs deep learning framework, comprising the following steps: S10: Acquire raw borehole data, perform coordinate system transformation and rotation correction, define a 3D bounding box and divide it into a regular voxel mesh, and generate the input tensor through interpolation. S20, shallow feature extraction is performed on the input tensor using a 3D convolutional layer to obtain a preliminary geometric feature map; S30, the preliminary geometric feature map is grouped and reorganized through 3D convolution operation, and scalar neurons are converted into vector capsules to explicitly encode the existence probability and spatial pose parameters of geological entities; S40 applies a capsule convolution module to perform multi-scale feature clustering on vector capsules; introduces a multi-head self-attention mechanism in the network bottleneck layer to calculate voxel correlations across the entire graph to capture long-distance geological structural correlations; and uses graph Laplacian sampling based on spectral theory for downsampling to preserve the topological skeleton information of geological bodies and obtain deep features. S50 employs an upsampling strategy combined with symmetrical hierarchical skip connections to restore the deep features to the original voxel resolution and outputs the conditional probability distribution of geological categories. S60, based on the conditional probability distribution, the network parameters are iteratively updated using a joint objective function to reconstruct the three-dimensional geological attribute field.

[0008] As a further preferred embodiment, in step S10, the rotation correction includes: Calculate the geometric center of all borehole coordinates and use it as the origin of the local Cartesian coordinate system; Based on the main structural orientation and azimuth of the study area The coordinates are rotated to make the principal axis of the voxel grid parallel to the formation strike.

[0009] As a further preferred embodiment, in step S30, the feature representation mechanism of the vector capsule includes: The modulus of the capsule vector represents the probability of the existence of a geological entity, and the range of the modulus is compressed to [0,1) by a nonlinear Squash function; The spatial pose information of geological entities is encoded using the orientation of capsule vectors, and the coupling coefficient is updated through a dynamic routing protocol during feature propagation.

[0010] As a further preferred embodiment, in step S40, the specific implementation of the graph Laplacian sampling is as follows: Construct an adjacency matrix W based on feature similarity and physical spatial distance; Calculate the normalized Laplace matrix based on the adjacency matrix. And perform feature decomposition on it; Low-frequency component nodes corresponding to the minimum eigenvalue are selected for retention, and high-frequency noise is filtered out to suppress the boundary step effect.

[0011] As a further preferred embodiment, in step S50, the feature fusion method of the skip connection is as follows: Establish a connection between the symmetrical levels of the encoder and decoder; Before splicing, the features output by the encoder are channel-adjusted to match the dimensions of the decoder capsules before splicing.

[0012] As a further preferred embodiment, in step S60, the joint objective function Loss due to capsule interval Mixed classification loss and rebuild regularization terms Weighted composition; The capsule interval loss Used to constrain capsule length and enhance the discriminative power of feature vectors in the latent space; The hybrid classification loss This method combines Dice loss and cross-entropy loss to address class imbalance in borehole data. The reconstruction regularization term By calculating the mean square error between the input voxels and the reconstructed voxels through the reconstruction branch, the model is forced to learn physically interpretable spatial features.

[0013] As a further preferred embodiment, the hybrid classification loss In this study, Dice loss is used to optimize the volume overlap between the predicted results and the true labels, and to enhance the sensitivity of small-volume geological bodies. The capsule interval loss An upper threshold is set. =0.9 and lower bound threshold =0.1.

[0014] As a further preferred embodiment, the capsule interval loss The calculation formula is:

[0015] The hybrid classification loss The calculation formula is:

[0016] The reconstruction regularization term The calculation formula is:

[0017] in, As an indicator function, when the k-th type of geological body exists in the input voxel =1, otherwise =0; and These are the upper and lower bound thresholds, respectively; The weighting coefficient; The L2 norm of the k-th output capsule vector represents the confidence (or probability) of the model predicting the existence of a specific type of geological body, while the direction of the vector encodes the instantiation attributes of the geological body, such as its spatial orientation and scale. Voxel representation The true label; The model predicts voxels. The probability value of belonging to a specific geological category; Indicates the smoothing term; and These represent the original 3D geological voxel blocks as input and the 3D geological voxel blocks as reconstructed by the network decoding, respectively. N The total number of three-dimensional spatial voxels involved in the loss calculation.

[0018] As a further preferred embodiment, in step S60, the iteratively updated training strategy further includes: A cosine annealing strategy is used to dynamically adjust the learning rate, so that the learning rate decays as a cosine function with each training round. A gradient accumulation strategy is introduced, and the parameters are updated every 4 backpropagation steps when the physical batch size is 1, so as to achieve the equivalent training effect of large batches. An early stopping mechanism is adopted, which monitors the average Dice coefficient on the validation set. When the validation performance does not improve after 30 consecutive training rounds, early stopping is triggered and the model weights are rolled back.

[0019] Secondly, this application provides a three-dimensional geological modeling system based on an improved UCAPs deep learning framework for implementing the method described in any one of the above statements, comprising: The data module is used for cleaning, coordinate transformation, and 3D voxelization encapsulation of borehole data; The model module integrates an improved UCAPs network topology, which includes a capsule convolutional layer, a multi-head self-attention layer, a graph Laplacian sampling layer, and a reconstruction branch. The training control module is used to configure the optimizer, dynamic learning rate scheduling, and early stopping mechanism.

[0020] Compared with the prior art, this application has the following significant advantages: (1) Significantly improves modeling accuracy: Experiments on real complex geological datasets in Jinan show that the overall accuracy of this method reaches 0.831, which is about 24.6% higher than that of ordinary Kriging interpolation; (2) Enhanced spatial consistency constraints: The vector representation and dynamic routing mechanism of capsule networks endow the model with implicit geological prior constraints, which effectively reduces prediction artifacts that are physically impossible to occur (such as suspended rock masses). (3) Breaking through the limitations of local features: The self-attention mechanism realizes the global perception of geological sedimentary patterns and can effectively infer the changing trend of unsampled areas at a distance based on known borehole data; (4) Improved topological structure preservation: The Thulaplatz sampling strategy avoids the loss of key geometric frameworks, making the generated stratigraphic boundaries sharper and more in line with natural sedimentary patterns. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the improved UCAPs modeling method that integrates self-attention mechanism and graph Laplacian sampling, as provided in the embodiments of this application. Figure 2 This is a neural network structure diagram of the improved UCAPs modeling method that integrates self-attention mechanism and graph Laplacian sampling provided in the embodiments of this application; Figure 3 This is a 3D voxel visualization result of the improved UCAPs modeling method that integrates self-attention mechanism and graph Laplacian sampling provided in the embodiments of this application; Figure 4 This is a cross-sectional visualization of the improved UCAPs modeling method that integrates self-attention mechanism and graph Laplacian sampling, provided in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] Please refer to Figure 1 This embodiment provides an improved UCAPs deep learning modeling method that integrates self-attention mechanism and graph Laplacian sampling. Its main steps are as follows: I. Data Preparation and 3D Voxelization (corresponding to step S1) High-quality data input is the cornerstone of ensuring the performance and convergence stability of deep learning models. Addressing the characteristics of geological borehole data, such as multi-source heterogeneity, sparse spatial distribution, and noise interference, this embodiment establishes a standardized data preprocessing pipeline. This pipeline covers key steps including data cleaning, coordinate space transformation, feature enhancement, and 3D voxel generation.

[0024] The borehole data is derived from engineering geological survey data and mainly includes the following structured data: a unique identifier (ID) for each borehole, borehole coordinates and depth, and internal layering information for each borehole, including bottom layer depth, formation thickness, and lithological description. Some data samples are shown in Table 1. Table 1 Drilling Data Table

[0025] To ensure data integrity and logical consistency, a rigorous data cleaning and quality control strategy was implemented. This included removing duplicate records from the database and eliminating invalid boreholes lacking crucial coordinate information. Outliers were checked based on geological sedimentary patterns, with a focus on correcting or removing illogical records such as bottom depths less than top depths or negative stratigraphic thicknesses. Furthermore, to address the subjective differences in lithological descriptions, a standardized dictionary was established, and the text was normalized and cleaned. For example, terms like "moderately weathered sandstone" and "strongly weathered sandstone" were uniformly mapped to standard lithological codes to eliminate semantic ambiguity.

[0026] The original borehole data used WGS84 latitude and longitude coordinates. Since latitude and longitude units are inconsistent with elevation units (meters), and geological bodies are typically distributed along structural trends, directly using the original coordinates is not conducive to gridding. Therefore, we converted the coordinates to a local Cartesian coordinate system with the center of the study area as the origin, and performed rotational correction along the main structural line direction:

[0027] in, Represents spatial coordinates in the original coordinate system. Represents the transformed local Cartesian coordinates. The geometric center of all borehole coordinates, The azimuth angle represents the main structural strike of the study area. By rotating the grid so that its principal axis is parallel to the stratigraphic strike, the proportion of invalid background voxels can be minimized, thus improving computational efficiency.

[0028] To adapt to the input requirements of the 3D UCaps model, the discrete borehole data needs to be converted into a regular voxel mesh. The specific steps are as follows: Step 1: Define the mesh space A bounding box covering the study area is defined and divided into a grid of size Length × Width × Height. Due to the complexity of the geological structure and computational resource constraints, the size of a single training sample is set to 32 × 32 × 32. For large-scale work areas, a sliding window strategy is used for slicing.

[0029] Step 2: Geological attribute interpolation and label generation To construct the Ground Truth (true label) required for supervised learning, we use a formation-based nearest neighbor interpolation method to initially densify the sparse borehole data.

[0030]

[0031] in, Indicates the position to be interpolated Predicted geological labels generated at the location. Represents the borehole observation dataset In the middle, with the point to be interpolated The nearest Geological labels for known sampling points. Indicates in the observation dataset In this context, the coordinates of the optimal neighborhood sampling points are determined by minimizing the weighted distance criterion. To weight the Euclidean distance, a larger weight is usually assigned to the vertical direction to reflect the characteristics of sedimentary bedding.

[0032] Step 3: Semantic Category Mapping To highlight the hydrogeological significance, the original fine-grained lithology is mapped into four types of engineering geological units: Permeable layer: This unit mainly consists of coarse-grained clastic rocks (such as coarse sandstone and conglomerate). These rocks have high primary porosity and good connectivity, serving as the main aquifer and infiltration channels in the groundwater circulation system; Aquitard: This mainly includes fine-grained or argillaceous cemented lithologies such as shale, mudstone, and siltstone. Due to its dense structure and extremely low permeability, this unit plays a significant role in blocking water in space; Reservoir / Aquifer: This category is specifically established for carbonate rocks. The special characteristic of this type of stratum is that it often has secondary porosity, fractures, or karst cavities, possessing extremely strong fluid storage potential; Background: The background class serves as engineering placeholders for the voxel grid, mainly including invalid areas within the study area, unclassified loose surface deposits, or other geological backgrounds not directly related to the modeling objective. In deep learning tasks, setting a background class is not only to balance the data distribution, but also to ensure the mathematical integrity of the 3D spatial mapping and prevent the model from making incorrect attribute inferences about edge regions. The specific mapping relationship is shown in Table 2.

[0033] Table 2 Mapping Correspondence

[0034] Step 4: 3D Data Augmentation To address the scarcity of borehole data, online 3D data augmentation is introduced during training. Random rotation of the data is performed along the Z-axis. Random rotation simulates minute changes in the strike of strata. Anisotropic scaling: performed separately in the horizontal and vertical directions. Random scaling by a factor of 100 simulates variations in formation thickness. Gaussian noise injection: added to the physical property channel. Gaussian white noise is used to improve the model's robustness to measurement errors. Random discarding is also employed: some known voxels in the input grid are randomly set to zero, forcing the model to learn to complete the model using contextual information.

[0035] II. Constructing an improved UCAPs model (corresponding to steps S2-S5) The network structure of the method in this embodiment is as follows: Figure 2 As shown, it specifically includes: 1. Shallow feature extraction layer (S2): To capture shallow geometric features, the data first passes through a feature extractor, which acts as the "stem" structure, after entering the network. This module utilizes standard 3D convolutional layers with BatchNorm and ReLU activation functions to map the original 3D input to a high-dimensional feature space.

[0036]

[0037]

[0038] in, Represents the original input tensor. This represents the intermediate feature tensor obtained after three-dimensional convolution operations. This represents the output feature map after batch normalization and ReLU activation function processing, which serves as the input for subsequent capsule layers. This represents a 3D convolution operator used to extract local features simultaneously across 3D spatial dimensions (depth, height, and width). This indicates the number of convolution kernels. This operation increases the number of input channels from 1 to 64, thereby mapping geological information to a high-dimensional feature space. This indicates the size of the convolution kernel, meaning that a sliding window of size 3 is used to extract features in all three dimensions of three-dimensional space. Indicates the step size. Indicates filling. This represents the three-dimensional batch normalization operation, used to accelerate network convergence and alleviate gradient problems in deep network training. This represents the modified linear unit activation function, which enhances the model's ability to represent complex geological structures by introducing nonlinear transformations.

[0039] The main purpose of this step is to extract shallow geometric features and reduce the computational load of subsequent capsule operations, by transforming the input tensor... Feature map converted to 64 channels This is in preparation for encapsulation.

[0040] 2. Construction of the master capsule layer (S3): To achieve the transformation from scalar to vector, the extracted feature map is then fed into the main capsule layer, which is the key interface for the model to transform from scalar neurons to vector capsules. The 3D convolution operation groups and reassembles the feature maps along the depth direction, forming capsules with 32 channels, each capsule being an 8-dimensional vector. These initial capsules not only encode the probability of the existence of geological features (vector magnitude), but also initially capture local texture and spatial pose information.

[0041]

[0042]

[0043] in, Indicates the process The high-dimensional intermediate feature tensor obtained after convolution. This represents the final generated primary capsule tensor. Denotes the dimension of the recombined tensor, where 32 represents the batch size, 32 represents the number of capsule channels, and 8 represents the vector dimension (dimension) of each capsule. These represent the depth, height, and width of a voxel in space. The core breakthrough of capsule networks lies in using vectors instead of scalars as the basic representation units of features, thus endowing neurons with explicit attribute expression capabilities. Signal transmission between capsules is not determined by a fixed weight matrix, but by dynamically calculated coupling coefficients. This dynamic routing mechanism allows the network to dynamically adjust connection paths based on the characteristics of the input data, thereby better handling geological deformation and viewpoint changes.

[0044] Input vector After transformation matrix Obtain the prediction vector High-density capsules It is a weighted sum of the prediction vectors:

[0045] Where the coupling coefficient From "routes" The result is obtained by Softmax normalization and iterative updates to maximize the consistency between the dot product of the predicted and output vectors.

[0046] in This is the logarithm of the prior probability. To compress the vector magnitude to... The interval represents the probability of an entity's existence. Since the direction represents the entity's attribute, we need to keep the vector direction unchanged. Therefore, we used the non-linear Squash function:

[0047] in, This indicates the number after activation. Output capsule vectors. This represents the intermediate state vector before entering the activation function, which is the total input after weighted aggregation of the prediction vectors from the lower-level capsules. Represents the input vector The L2 norm (modulus) of the model. By applying this Squash function, the model can transform abstract feature strengths into intuitive geoscientific probability distributions, providing a quantitative basis for the final three-dimensional geological reconstruction.

[0048] 3. Multi-head self-attention and graph sampling (S4): The capsuled data flow is directed to an encoder consisting of six stacked capsule convolutional modules. Each module integrates 3D capsule convolution, dynamic routing algorithms, and batch normalization, and uses an iterative protocol algorithm to achieve feature clustering, ensuring spatial consistency between the "parts" and the "whole." Geological structures often exhibit strong nonlocality; for example, fault extensions may span the entire work area, or sedimentary facies changes may be correlated over long distances. Traditional 3D convolutional operations are limited by fixed local receptive fields, making it difficult to effectively capture these long-distance spatial dependencies. To address this, a multi-head self-attention mechanism is integrated into the encoder's bottleneck layer. By mapping features to queries, keys, and values, voxel associations across the entire graph are computed.

[0049] in, The dimension of the feature vector is represented by dividing by Scaling is performed to prevent the dot product result from becoming too large, which could lead to gradient vanishing and ensure numerical stability during training. This represents the normalized exponential function, used to transform affinity scores into a weighted probability distribution ranging from [0, 1]. This represents the final attention output. It is a vector of values. The weighted aggregation allows the model to explicitly incorporate the context of other relevant geological units across the entire map when processing the current voxel. This module overcomes the limitations of the local receptive field of the convolution kernel, enabling the model to establish interactions between long-distance geological features such as fault extension and sedimentary facies transitions, and introduces global context during feature aggregation.

[0050] In terms of sampling strategy, this model abandons traditional pooling and adopts graph Laplacian sampling based on spectral graph theory. By performing eigenvalue decomposition on the adjacency matrix, it retains low-frequency structural information representing the geological body's framework, effectively mitigating the loss of topological structure caused by traditional sampling. An adjacency matrix W is constructed, with weights... Based on feature similarity and spatial distance:

[0051] in, Represents the nodes in the graph With nodes Edge weights (similarity scores) between them. This represents the spatial location vector of a voxel node. This represents the feature vector of a voxel node at this level. These represent the scale parameters (hyperparameters) for spatial distance and feature distance, respectively, and are used to adjust the model's sensitivity to spatial location and feature differences.

[0052] Calculate the normalized Laplace matrix:

[0053] in, This represents the normalized graph Laplacian matrix. It is a core operator characterizing the smoothness and structural features of graph signals. This represents the inverse square root of the degree matrix, used to standardize the adjacency weights and eliminate the influence of node degree differences on spectral analysis. Eigenvalue decomposition is performed, and nodes with the smallest corresponding eigenvalues ​​(low-frequency components) are selected and retained. This is equivalent to preserving the main structural information of the graph (i.e., the skeleton of the geological body) while filtering out high-frequency noise.

[0054] 4. Decoding and skip connections (S5): The data is then processed by a decoder to restore spatial resolution, generating the final voxel-level prediction. We employ a strategy combining trilinear interpolation upsampling and 3D convolution:

[0055] in, Indicates the decoder's first The output feature tensor of the layer. This represents the input features from the previous decoder or bottleneck layer. This represents a trilinear interpolation upsampling operation. The voxel resolution is gradually restored by doubling the spatial dimension. To compensate for the spatial information lost during downsampling, we establish skip connections between the symmetric layers of the encoder and decoder. Unlike the direct concatenation of U-Net, we perform channel adjustment on the encoder features before concatenation to match the capsule dimension of the decoder.

[0056] in, This indicates a concatenation operation, which merges the current decoded feature with the corresponding encoded feature along the channel dimension. This represents the corresponding symmetric layer feature in the encoder. This represents the channel adjustment function. Since the encoder outputs capsule vectors, this transformation is used to adjust the number of channels and the vector dimension to ensure an exact match with the decoder's dimensional structure. Furthermore, the network introduces a reconstruction branch that uses fully connected layers to reconstruct the original input from the predicted capsules, serving as a regularization term to force the vector encoding to specific instantiation parameters. This branch utilizes fully connected layers to reconstruct the original input from the predicted capsule vectors. Reconstructing the original input image :

[0057] in, This represents a reconstruction decoder composed of multiple fully connected layers. It maps the high-dimensional abstract capsule properties back to the original voxel space, acting as a regularization term to force the network to learn richer geological properties. Ultimately, the model outputs a conditional probability distribution of k geological categories, achieving a digital reconstruction of hidden underground geological bodies.

[0058] III. Model Training and Parameter Optimization (corresponding to step S6) 1. Loss function configuration: To achieve accurate segmentation of the geometric shape of geological bodies and ensure that capsule vectors effectively encode spatial features, this embodiment constructs a multi-task joint optimization objective. The objective function... By weighting and combining the capsule interval loss (Margin Loss), the classification loss (Classification Loss), and the reconstruction regularization term (Reconstruction Loss), the model can capture the physical spatial properties of geological bodies while maintaining its class discrimination ability. Specifically, the joint objective function is formalized as follows:

[0059] in, Capsule spacing loss is used to enhance the discriminative power of feature vectors in the latent space; This is a hybrid classification loss used to improve the segmentation accuracy of the model under class imbalance conditions; To reconstruct the loss, the input voxels can be reconstructed by constraining the feature vector, thus achieving regularized encoding of the capsule features. The design concepts and mathematical expressions of each loss function are explained in detail below.

[0060] 1) Margin Loss The core idea of ​​capsule networks is to represent the probability of entity existence through vector magnitude, while the vector direction encodes the spatial and attribute information of the entity. To enhance the discriminative power of feature vectors in the latent space, this study introduces a margin loss specifically designed for capsule networks. This loss borrows from the maximum margin classification idea of ​​Support Vector Machines (SVM), constraining the mapping relationship between capsule magnitude and class existence probability, thereby encouraging close clustering of similar features and significant spacing between dissimilar features. For the ... For geological-like bodies, the loss function is defined as follows:

[0061] in, As an indicator function, when the k-th type of geological body exists in the input voxel =1, otherwise =0. and These are the upper and lower bound thresholds, respectively. This study sets... =0.9, =0.1. This means that when a specific geological body is present, the modulus of the corresponding capsule is forced to tend to be greater than 0.9; otherwise, it is compressed to less than 0.1. This is the weighting factor (usually set to 0.5), used to reduce the proportion of negative samples, i.e., non-existent classes, in the total loss, and to prevent gradient dominance caused by too many background voxels in the early stages of training.

[0062] 2) Classification Loss (Mixed Classification Loss) Geological borehole data commonly suffers from severe class imbalance, with thin coal seams or lenses of significant economic value accounting for a very small percentage of the total volume. Using only traditional cross-entropy loss can easily lead to overfitting the model to the background class, neglecting the characteristics of rare geological bodies. Therefore, this study constructs a hybrid classification loss function combining Dice Loss and Cross Entropy (CE) Loss:

[0063] in, Voxel representation The true labels are typically represented using one-hot encoding in multi-class classification tasks. The model predicts voxels. The probability value of belonging to a specific geological category. The term represents a smoothing term, which prevents the denominator from being zero and increases numerical stability during training. Dice Loss directly optimizes the volume overlap between the predicted result and the true label, showing a significant advantage in the segmentation of small geological objects; while CE Loss ensures classification accuracy from the perspective of pixel-level probability distribution. The combination of the two effectively improves the model's generalization ability under extremely imbalanced sample distributions, and also enhances the overall stability and generalization of predictions.

[0064] 3) Reconstruction Loss (Reconstruction Regularization) Capsule vectors should not only encode the probability of category existence but also carry the instantiated spatial features of the geological body, such as stratigraphic thickness, dip angle, and texture structure. Therefore, this study introduces a decoding branch at the end of the capsule network to map the latent vectors back to the input voxels, achieving the reconstruction of the original geological body. The optimization objective of this branch is the input voxels... With reconstructed voxels Mean squared error (MSE) between:

[0065] This loss term, used as regularization, does not directly participate in classification, but by forcing the capsule vectors to retain rich information from the input voxels, it promotes the network to learn more physically interpretable spatial features. Furthermore, the model incorporating reconstruction loss significantly outperforms the model relying solely on classification loss in segmentation accuracy for small-volume geological bodies and complex geometries, indicating that the capsule vectors successfully capture the spatial and structural information of the geological bodies.

[0066] 2. Training strategy: The Adam algorithm was used to optimize the model parameters. Compared to stochastic gradient descent, the Adam algorithm combines the first-order moment estimation of the momentum method with the second-order moment estimation of RMSProp, enabling it to adaptively adjust the learning rate for different parameters. This characteristic makes it particularly suitable for handling sparse gradient problems, commonly found in voxelized geological data, and non-stationary objective functions. The key hyperparameter settings during the optimization process are as follows: Parameter update rules:

[0067] in, This represents the updated network parameters (weights). This represents the network parameters at the current moment. The learning rate determines the step size for parameter updates. This represents the first-order moment estimate (i.e., the mean of the gradient) after bias correction, used to accelerate convergence using momentum. This represents the second-order moment estimate after bias correction, used to adjust the learning rate scaling for each parameter. This represents the numerical stability constant, used to prevent the denominator from being zero.

[0068] Hyperparameter: Initial learning rate Weight decay is set to To prevent overfitting.

[0069] To achieve rapid convergence in the early stages of training and fine-tune the model later, a cosine annealing strategy is employed. The core idea is to use a periodically, smoothly decreasing learning rate to allow the network to quickly explore the parameter space with a large learning rate in the early stages of training, thus accelerating convergence. In the later stages, the learning rate is gradually reduced to refine the model parameters, preventing the model from getting stuck in local optima and improving its generalization ability and convergence stability. Under this strategy, the learning rate decays cosinely with each epoch, avoiding getting trapped in local minima.

[0070] The physical meaning and settings of each parameter in the formula are as follows: Indicates the first The current learning rate used by the model at each epoch is a variable that changes dynamically as the training process progresses. This represents the upper bound of the learning rate, i.e., the initial learning rate. This represents the lower bound of the learning rate, i.e., the preset minimum learning rate. This represents the maximum number of training epochs or a complete cycle of cosine annealing. In this experiment, the maximum number of training epochs was set to 200, and the model completed a smooth transition from the maximum learning rate to the minimum learning rate within this cycle. Through this scheduling method, the learning rate is kept at a high level in the early stage of training to quickly reduce the loss, and then decays smoothly, which helps the model escape local minima and eventually converge to a better global solution.

[0071] Given the significant memory consumption of 3D capsule networks, directly increasing the batch size is often limited by hardware constraints. This study sets the physical batch size to 1 and introduces a gradient accumulation strategy, updating parameters every four backpropagation steps. Mathematically, this is equivalent to training with a batch size of 4, ensuring both statistical stability of gradient estimation and adapting to memory constraints. To prevent overfitting, this study employs an early stopping mechanism. Specifically, by continuously monitoring the average Dice coefficient on the validation set, early stopping is triggered when validation performance fails to improve for 30 consecutive epochs, automatically rolling back to the best-performing model weights, thus ensuring both efficient and stable training.

[0072] IV. Model Reasoning and Evaluation During the inference phase, the voxelized tensor of the region to be modeled is input into the trained model. The model outputs a probability vector for each voxel belonging to categories such as permeable layer and impermeable layer. The category with the highest probability is taken as the final attribute label for that voxel. Finally, a 3D geological model is generated using the voxels for visualization. The visualization results are shown below. Figure 3 , Figure 4 As shown.

[0073] To verify the effectiveness of the model, this study compared the proposed complete method with existing mainstream geological modeling methods, including traditional interpolation methods (Nearest Neighbor NN, Inverse Distance Weighted IDW, Ordinary Kriging OK) and mainstream deep learning baselines (3D U-Net, Original 3D CapsNet). Table 3 summarizes the performance metrics of each method on the test set.

[0074] Table 3 Summary of Performance Indicators

[0075] Experimental results show that all deep learning-based methods significantly outperform traditional geostatistical methods across all metrics. For example, the overall accuracy of ordinary kriging (OK) is 0.667, while the complete method proposed in this study achieves 0.831, an improvement of approximately 24.6%. This significant difference reveals the limitations of traditional methods in handling non-stationary geological processes: kriging interpolation relies on a linear unbiased estimate constructed from a variogram, primarily based on smooth inference using Euclidean geometric distance; whereas neural networks can learn complex sedimentary patterns and nonlinear spatial correlations implicit in stratigraphic sequences from sparse data through multi-layered nonlinear mapping.

[0076] Under identical experimental conditions, the original UCAPs model outperformed the 3D U-Net (0.634) in average Dice (0.645) and also showed a significant improvement in overall accuracy (OA) (0.756 vs 0.721). This performance gain is mainly attributed to the unique vectorized representation mechanism of the capsule network. Unlike the scalar activation of CNNs, capsule vectors can explicitly encode the spatial pose parameters of geological entities. When strata undergo folding or tilting deformation, the rotational isovariability of the capsule network enables it to more robustly maintain the geometric consistency of the geological structure.

[0077] The integrated method combining self-attention and graph Laplacian sampling achieved state-of-the-art performance, with an average Dice of 0.707 and an overall accuracy exceeding 83.1%. Particularly noteworthy is the improvement in the Dice coefficient of the most challenging element, "reservoir" identification, which increased from 0.537 in 3D U-Net to 0.586. Considering that limestone often exhibits discrete lenticular distributions in strata and has a very low sample size (long-tailed distribution), this improvement demonstrates the effectiveness of the self-attention mechanism in capturing karst heterogeneity and the advantage of the graph sampling strategy in preserving the topology of fractured geological bodies.

[0078] To deconstruct the marginal contribution of each innovative module in the UCAPs framework to the overall model performance, this embodiment designed a systematic ablation experiment. While maintaining consistent baseline parameters, self-attention (Self-Attn), graph Laplacian sampling (Graph-Pool), and reconstruction loss (Rec-Loss) were gradually introduced. The experimental results are shown in Table 4.

[0079] Table 4 Ablation Experiment Results

[0080] Comparing configurations A and B, it is evident that the introduction of the self-attention mechanism resulted in the largest single-item performance gain (Dice +2.2%). Geological structures such as fault extensions and stratigraphic pinch-outs often exhibit long-distance spatial dependencies. Traditional convolution operations are limited by local receptive fields, making it difficult to capture such global correlations; while the self-attention mechanism, by calculating voxel similarity across the entire image, effectively establishes the interaction of distant geological features, significantly improving the model's ability to infer geological continuity. Configuration C shows that using graph Laplacian sampling instead of traditional max pooling improved Dice by approximately 1.3%. Max pooling tends to retain strong texture features while ignoring spatial structure, easily leading to blurred geological boundaries. Conversely, graph sampling, based on spectral theory, retains low-frequency structural components. Qualitatively, this effectively suppresses the "staircase effect" common in voxelized meshes, resulting in sharper stratigraphic boundaries that conform to geological sedimentary patterns. Although the direct performance improvement from reconstruction loss is relatively small (+0.8%), training dynamics analysis reveals that the introduction of... Subsequently, the oscillations of the validation set loss curve decreased significantly, and the convergence process became more stable. This indicates that the reconstruction task forced the capsule vectors to encode richer geological attributes such as density and lithological texture, which played a strong regularization role and prevented the model from overfitting to sparse borehole data.

[0081] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A modeling method for an improved UCAPs deep learning framework, characterized in that, Includes the following steps: S10: Acquire raw borehole data, perform coordinate system transformation and rotation correction, define a 3D bounding box and divide it into a regular voxel mesh, and generate the input tensor through interpolation. S20, shallow feature extraction is performed on the input tensor using a 3D convolutional layer to obtain a preliminary geometric feature map; S30, the preliminary geometric feature map is grouped and reorganized through 3D convolution operation, and scalar neurons are converted into vector capsules to explicitly encode the existence probability and spatial pose parameters of geological entities; S40 applies a capsule convolution module to perform multi-scale feature clustering on vector capsules; a multi-head self-attention mechanism is introduced in the network bottleneck layer to calculate voxel correlations across the entire image to capture long-distance geological structural correlations. Downsampling was performed using graph Laplace sampling based on spectral theory to preserve the topological skeleton information of the geological body and obtain deep features; S50 employs an upsampling strategy combined with symmetrical hierarchical skip connections to restore the deep features to the original voxel resolution and outputs the conditional probability distribution of geological categories. S60, based on the conditional probability distribution, the network parameters are iteratively updated using a joint objective function to reconstruct the three-dimensional geological attribute field.

2. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S10, the rotation correction includes: Calculate the geometric center of all borehole coordinates and use it as the origin of the local Cartesian coordinate system; Based on the main structural orientation and azimuth of the study area The coordinates are rotated to make the principal axis of the voxel grid parallel to the formation strike.

3. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S30, the feature representation mechanism of the vector capsule includes: The modulus of the capsule vector represents the probability of the existence of a geological entity, and the range of the modulus is compressed to [0,1) by a nonlinear Squash function; The spatial pose information of geological entities is encoded using the orientation of capsule vectors, and the coupling coefficient is updated through a dynamic routing protocol during feature propagation.

4. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S40, the specific implementation of the graph Laplacian sampling is as follows: Construct an adjacency matrix W based on feature similarity and physical spatial distance; Calculate the normalized Laplace matrix based on the adjacency matrix. And perform feature decomposition on it; Low-frequency component nodes corresponding to the minimum eigenvalue are selected for retention, and high-frequency noise is filtered out to suppress the boundary step effect.

5. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S50, the feature fusion method of the skip connection is as follows: Establish a connection between the symmetrical levels of the encoder and decoder; Before splicing, the features output by the encoder are channel-adjusted to match the dimensions of the decoder capsules before splicing.

6. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S60, the joint objective function Loss due to capsule interval Mixed classification loss and rebuild regularization terms Weighted composition; The capsule interval loss Used to constrain capsule length and enhance the discriminative power of feature vectors in the latent space; The hybrid classification loss This method combines Dice loss and cross-entropy loss to address class imbalance in borehole data. The reconstruction regularization term By calculating the mean square error between the input voxels and the reconstructed voxels through the reconstruction branch, the model is forced to learn physically interpretable spatial features.

7. The modeling method of the improved UCAPs deep learning framework as described in claim 6, characterized in that, The hybrid classification loss In this study, Dice loss is used to optimize the volume overlap between the predicted results and the true labels, and to enhance the sensitivity of small-volume geological bodies. The capsule interval loss An upper threshold is set. =0.9 and lower bound threshold =0.

1.

8. The modeling method of the improved UCAPs deep learning framework as described in claim 6, characterized in that, The capsule interval loss The calculation formula is: The hybrid classification loss The calculation formula is: The reconstruction regularization term The calculation formula is: in, As an indicator function, when the k-th type of geological body exists in the input voxel =1, otherwise =0; and These are the upper and lower bound thresholds, respectively; The weighting coefficient; Let L2 norm represent the k-th output capsule vector; Voxel representation The true label; The model predicts voxels. The probability value of belonging to a specific geological category; Indicates the smoothing term; and These represent the original 3D geological voxel blocks as input and the 3D geological voxel blocks as reconstructed by the network decoding, respectively. N The total number of three-dimensional spatial voxels involved in the loss calculation.

9. The modeling method of the improved UCAPs deep learning framework as described in claim 1, characterized in that, In step S60, the iteratively updated training strategy further includes: A cosine annealing strategy is used to dynamically adjust the learning rate, so that the learning rate decays as a cosine function with each training round. A gradient accumulation strategy is introduced, and the parameters are updated every 4 backpropagation steps when the physical batch size is 1, so as to achieve the equivalent training effect of large batches. An early stopping mechanism is adopted, which monitors the average Dice coefficient on the validation set. When the validation performance does not improve after 30 consecutive training rounds, early stopping is triggered and the model weights are rolled back.

10. A three-dimensional geological modeling system based on an improved UCAPs deep learning framework, characterized in that, For implementing the method as described in any one of claims 1 to 9, comprising: The data module is used for cleaning, coordinate transformation, and 3D voxelization encapsulation of borehole data; The model module integrates an improved UCAPs network topology, which includes a capsule convolutional layer, a multi-head self-attention layer, a graph Laplacian sampling layer, and a reconstruction branch. The training control module is used to configure the optimizer, dynamic learning rate scheduling, and early stopping mechanism.

Images