Multi-head attention model training method fusing geological rules

Abstract

The invention relates to a geological analysis technology, and discloses a multi-head attention model training method fused with geological rules, which is used for improving the scientificity and accuracy of geological body model prediction and improving the interpretability of a geological analysis process. According to the scheme, firstly, feature vectors of geological attribute data and geological coordinate data are spliced to obtain a fusion feature vector, then spatial locality and directivity are considered at the same time, geomorphic multi-head attention calculation is conducted on the fusion feature vector, and an attention feature sequence is obtained; then extracting multi-scale features from the input geological map, constructing a cross-fault weight mask matrix based on fault constraints, and obtaining comprehensive features through fusion; calculating a loss value by adopting a joint loss function containing prediction loss and fault non-penetrability constraint terms, and updating model parameters through a back propagation algorithm to complete model training; and finally, drawing an attention thermodynamic diagram for visual display, and superposing the attention thermodynamic diagram with the three-dimensional geologic model.

Description

Technical Field

[0001] The present invention relates to geological analysis technology, and in particular to a multi-head attention model training method integrating geological rules. Background Art

[0002] Accurately predicting the distribution and structure of geological bodies is crucial in geological exploration and resource development. Traditional geological body prediction methods rely primarily on the experience of geologists and simple mathematical models. These methods often struggle to handle complex geological data and variable geological structures, resulting in low prediction accuracy. With the development of deep learning technology, neural network-based methods have gradually been applied to geological data processing. However, existing deep learning models in the geological field still face problems such as poor interpretability and insufficient integration of domain knowledge, as shown in the following examples: Black box characteristics: The decision logic of the neural network lacks transparency, making it difficult for geological engineers to verify whether the model conforms to geological laws (such as fault cutting relationships). The results generated by this method have poor correlation with the geological background laws.

[0003] Weak consistency in geological operations: relying solely on data-driven approaches may generate structures that violate geological principles (such as stratum inversion). Summary of the Invention

[0004] The technical problem to be solved by the present invention is to propose a multi-head attention model training method that integrates geological rules to improve the scientificity and accuracy of geological body model prediction and enhance the interpretability of geological analysis process.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: A multi-head attention model training method integrating geological rules includes the following steps: S1. The three-dimensional geological coordinate data is converted into a high-dimensional vector by Fourier eigenmapping, and the geological attribute data is encoded into a feature vector by an embedding layer and a linear transformation; the high-dimensional vector and the feature vector are then concatenated and fused to obtain a fused feature vector; S2. Based on the fused feature vector, a sliding window is used for attention processing, while introducing a relative position bias calculated based on the relative position of the coordinates within the window. By extracting the directional characteristics of the geological structure and calculating the directional weight matrix, anisotropic attention heads are assigned according to the directional characteristics of the geological structure, obtaining an attention feature sequence that contains both spatial locality and directional characteristics. S3. Perform multi-scale convolution on the input geological map using a convolutional neural network to extract multi-scale feature data. Traverse the fault lines represented by vectors and calculate the orientation relationship based on the spatial distance and angle between the three-dimensional geological coordinates and the fault lines to generate a cross-fault weight mask matrix that reflects the fault constraints. Fusion the multi-scale feature data, the attention feature sequence, and the cross-fault mask matrix to obtain comprehensive features. S4. Using the comprehensive features as the input of the model, calculating the loss value using a joint loss function including the prediction loss and the fault impenetrability constraint term, updating the model parameters using a backpropagation algorithm, and completing the training of the geological body prediction model; S5. Based on the input geological data, the trained geological body prediction model is used to obtain the attention weight matrix, which is normalized by the softmax function. The heat map is drawn and superimposed with the 3D geological model.

[0006] Furthermore, in step S1, the geological attribute data includes but is not limited to: lithology, fault, and lithofacies data.

[0007] Furthermore, in step S2, the attention processing using a sliding window includes: Define the sliding window radius, and for the sequence representation of the fused feature vector, calculate the attention scores between the elements at each position in the sequence and the elements at other positions in its neighborhood. For the element at each position, a weighted sum is performed based on the attention scores of the elements at other positions in its neighborhood and the element value at the corresponding position to obtain the output sequence after attention processing.

[0008] Furthermore, in step S2, the introduction of the relative position offset calculated based on the relative position of the coordinates within the window refers to: In attention processing using a sliding window, when calculating the attention score between an element at each position in the sequence and elements at other positions in its neighborhood, the position bias calculated by the relative position of the coordinates within the window is taken into account.

[0009] Furthermore, in step S2, the direction weight matrix is ​​calculated using a direction cosine matrix according to the strike and dip of the geological structure.

[0010] Furthermore, in step S3, performing multi-scale convolution on the input geological map by using a convolutional neural network to extract multi-scale feature data includes: First, the input geological map is processed by Patch Embedding through 16×16 convolution to obtain a feature sequence; Then process it at three scales in turn: Scale 1 takes the original Figure 1 / 4 resolution, using 8×8 window multi-head attention combined with a maximum pooling operation with a stride of 2 to extract the local feature L1; Scale 2 takes the original Figure 1 / 8 resolution, using a 4×4 window of multi-head attention combined with a maximum pooling operation with a stride of 2 to extract the local feature L2; Scale 3 takes the original Figure 1 / 16 resolution, global feature L3 is obtained through global attention; Next, L3 is upsampled and initially fused with L2. After adjusting the channel through 1×1 convolution, the initial fused features after channel adjustment are upsampled again and fused with L1 to complete the multi-scale feature integration.

[0011] Furthermore, in step S3, the fault line represented by the traversal vector is traversed, and the orientation relationship is calculated based on the spatial distance and angle between the three-dimensional geological coordinates and the fault line to generate a cross-fault weight mask matrix reflecting the fault constraint, including: Create an all-1 mask matrix: Create an all-1 matrix of dimension N×N, where N is the dimension of geological location data; Traverse fault line data: Traverse the fault lines in the geological map one by one to locate all areas where physical constraints need to be applied; Calculate the coordinate and orientation relationship: for each geological coordinate point, calculate its spatial orientation relative to the currently traversed fault line; Generate cross-fault mask: Based on the orientation calculation results, the positions in the initial mask matrix that do not conform to geological rules are set to 0 to form a cross-fault weight mask matrix that reflects the fault constraints.

[0012] Furthermore, in step S5, based on the input geological data, the trained geological body prediction model is used to obtain an attention weight matrix, the attention weight matrix is ​​normalized by a softmax function, a heat map is drawn, and the heat map is superimposed and output with the three-dimensional geological model, including: The geological data is input into the trained geological volume prediction model. The attention mechanism within the model assigns attention weights to elements at different positions in the input data. These weights constitute the attention weight matrix. The obtained attention weight matrix is ​​normalized using the softmax function; According to the normalized attention weight matrix, each value in the matrix is ​​mapped to the corresponding color in the selected color mapping scheme; Use Python's drawing library to draw and obtain a heat map; Align the data in the thermal map with the data in the 3D geological model; The drawn thermal map is superimposed on the 3D geological model in a color-coded form.

[0013] The beneficial effects of the present invention are: (1) Accurate capture of geological features based on the geostructural attention mechanism: The three-dimensional geological coordinates are converted into high-dimensional vectors through Fourier feature mapping, and the geological attributes are embedded in the feature splicing to achieve joint encoding of coordinate space and geological semantics. This can improve the representation ability of spatial features and provide rich geological feature semantic information, laying the foundation for accurate prediction of the model.

[0014] (2) Through the direction-aware attention mechanism and the anisotropic kernel function, the directional characteristics of the geological structure are accurately captured, and the adaptability of the model to anisotropic properties such as fault strike and rock layer dip is improved: the azimuth encoding is used to convert the geological structure direction into a feature vector, and the anisotropic kernel is combined to allocate independent attention heads to different directions, so that the model can adaptively adjust the attention weights of different directions and enhance the modeling ability of the spatial orientation relationship of the geological body.

[0015] (3) Physically constrained attention and cross-fault mask matrices are introduced to transform geological rules such as fault impenetrability into attention calculation constraints, thus avoiding the generation of prediction results that violate geological principles from a mechanism perspective: by traversing the fault line to generate a mask matrix, unreasonable associations are shielded in the attention calculation, and the loss function constraint is used to force the model to follow the geophysical laws, ensuring that the prediction results are consistent with the geological rules.

[0016] (4) A multi-scale attention pyramid structure is adopted to achieve cross-scale information integration from meter-level joints to hundred-meter-level faults through feature extraction and upsampling fusion at different resolutions: by layering geological features at different scales, the model can capture the details of the microscopic joint network and grasp the macroscopic faults, thereby improving the hierarchical analysis capability of complex geological structures.

[0017] (5) By visually overlaying the attention heat map with the three-dimensional geological model, a geological interpretation system that can intuitively trace the model decision logic is constructed: the attention weight matrix is ​​normalized to generate a heat map, which is spatially overlaid with the three-dimensional model, and the key areas of the model's attention are highlighted with color coding, providing geological engineers with a transparent model decision-making auxiliary tool.

[0018] (6) Based on a joint loss function training system that includes prediction loss and geological constraints, balance data-driven fitting and geophysical rule constraints: Through the joint loss function, the coordinated optimization of data fitting and physical laws is achieved, so that the model follows geological prior knowledge while learning data characteristics, thereby improving the model's generalization ability and prediction reliability in complex geological scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the multi-head attention model training method that integrates geological rules in the present invention.

[0020] Figure 2 Flowchart for processing the splicing and fusion of high-dimensional vectors of geological coordinates and characteristic vectors of geological attributes.

[0021] Figure 3 Flowchart of attention processing for sliding windows.

[0022] Figure 4 Flowchart for the calculation of relative position offset.

[0023] Figure 5 Computational flowchart for direction-aware attention.

[0024] Figure 6 Flowchart for the generation of cross-slice masks. DETAILED DESCRIPTION

[0025] This paper aims to provide a multi-head attention model training method that integrates geological rules, improving the scientificity and accuracy of geological model predictions and enhancing the interpretability of geological analysis processes. Its core concept is to deeply integrate geological prior knowledge with the Transformer architecture through the systematic design of a "geostructural attention mechanism," thereby constructing an interpretable model that combines data-driven capabilities with geophysical constraints. The following innovative approaches are employed in this solution: (1) Optimization of geological structure-oriented attention mechanism architecture: By introducing azimuth encoding technology and anisotropic kernel functions, a refined model is constructed for typical stratigraphic directional elements such as fault strike and rock layer dip. This enables the model to accurately capture the spatial orientation of geological bodies. By encoding azimuth features and quantifying geological directional information, the model possesses highly sensitive perception and analysis capabilities when faced with geological data with complex directionality, ensuring the accuracy of geological feature extraction and analysis, and effectively avoiding deviations caused by directional misjudgment.

[0026] Fundamental physical rules in geology, typified by fault impenetrability, are transformed into attention mask constraints. During model execution, this mask acts as a precise "navigator," fundamentally limiting model output to adherence to established geological principles. This prevents excessive model freedom from producing results that violate common sense geophysical principles, ensuring that model behavior is deeply aligned with real geological laws and enhancing the reliability and scientific nature of model output.

[0027] (2) Interpretability strategy for deep embedding of geological features: Leveraging advanced Fourier eigenmapping algorithms, we perform feature conversion on highly specialized and semantically complex geological parameters such as lithology and joint occurrence. This technology transforms previously abstract, semantic geological information difficult to directly process into efficiently processed digital feature vectors, successfully building a bridge between geological expertise and intelligent algorithms, breaking down knowledge barriers and enabling the seamless flow and precise utilization of geological information within the model.

[0028] An innovative approach proposes a fusion of meter-scale joint analysis and hundred-meter-scale fault systems, creating a collaborative framework for cross-scale geological analysis. By integrating multi-scale information, the model not only focuses on the fine structure of microscopic joints but also captures the overall dynamics of macroscopic fault systems. This allows for a deep integration of microscopic details and macroscopic patterns within the same model, providing strong support for the comprehensive and accurate restoration of complex geological structures.

[0029] (3) Establishment of a joint driving training system integrating geological rules: An innovative joint-driven training framework has been constructed that combines data fitting with adherence to physical laws. This design introduces the inherent constraints of geophysical laws into model training. During the optimization process, the model not only pursues goodness of fit at the data level, but also ensures that the output closely matches geological reality. This accurately locates the balance between mathematical optimization and geological authenticity, ensuring the practicality and scientific nature of the geological model.

[0030] (4) Dynamic implementation mechanism of direction perception: Based on the real-time dynamic coupling of geomechanical parameters and attention weights, an intelligent weight distribution adjustment system is constructed. When processing geological data, the model can dynamically adjust the distribution of attention weights based on real-time feedback from geomechanical properties. Through this adaptive adjustment mechanism, the model can focus on key geological features, accurately capturing important information from geological data. This significantly improves the model's adaptability and analytical accuracy to complex and changing geological features, laying a solid foundation for precise engineering geological analysis.

[0031] (5) Construction of a visualization-enhanced geological model interpretation system: By integrating attention heatmaps with geological models, this technology provides geological engineers with a visual window into the model's internal operating logic, transforming the previously mysterious "black box" model into a transparent and visual decision-making tool. By visually presenting the model's attention distribution and the basis for decision-making, it significantly improves the interpretability of geological analysis processes, enhances the credibility of model outputs, and provides a solid foundation for geological engineering practice.

[0032] In specific implementation, the process of the multi-head attention model training method integrating geological rules provided by the present invention can be found in Figure 1 , which includes the following implementation steps: S1. Geological location characteristics: In this step, the three-dimensional geological coordinate data is converted into a high-dimensional vector through Fourier eigenmapping, and the geological attribute data is encoded into a feature vector through an embedding layer and linear transformation; then the high-dimensional vector and the feature vector are spliced ​​and fused to obtain a fused feature vector.

[0033] Among them, after the three-dimensional geological coordinate data is converted into a high-dimensional vector through Fourier characteristic mapping, the geological attributes (such as lithology, faults, lithofacies, etc.) are combined as additional features and embedded and spliced ​​with the coordinates. The process is shown in Figure 2 Specifically, the following steps are performed: Coordinate branch processing: 3D coordinates are converted to 20-dimensional vectors using Fourier eigenmapping, and then linearly upscaled to 32 dimensions to obtain coord_feat. Geological attribute branch processing: Discrete geological attributes (such as lithology and weathering degree) are mapped to 16-dimensional vectors using an embedding layer. After being combined with other continuous attributes (such as RQD value and joint occurrence), the vector is linearly upscaled to 32 dimensions to obtain geo_feat. Feature fusion: coord_feat and geo_feat are concatenated into a 64-dimensional fused feature vector, which serves as the input for subsequent attention calculations.

[0034] S2. Geo-structured multi-head attention calculation: In this step, the multi-head attention calculation of the geological structure needs to consider both spatial locality and directionality. Specifically, based on the fused feature vector, a sliding window is used for attention processing, while a relative position bias is introduced based on the relative position of the coordinates within the window. By extracting the directional characteristics of the geological structure and calculating the directional weight matrix, anisotropic attention heads are assigned according to the directional characteristics of the geological structure, obtaining an attention feature sequence that combines spatial locality and directional characteristics.

[0035] The process of sliding window attention processing can be found in Figure 3 , which includes the following processes: Input sequence: The input sequence is represented as X=x1,x2,…,x n , where x i Represents each position element in the geological position information sequence to be processed. These elements contain various geological characteristic data, such as coordinates, lithology, and other information. Subsequent operations will be carried out based on these position elements.

[0036] Set the window radius r: According to the requirements of geological spatial correlation and computational complexity, a suitable window radius r is pre-set. This radius determines the window radius of each position x. i The range of neighboring areas that are focused on during attention calculation allows the calculation to be focused locally, reducing the high complexity problems caused by global calculations and better adapting to geological space characteristics.

[0037] For each position x i , define the neighborhood: the neighborhood range is defined as the range from x i −r to x i +r means that other positions in this range will participate in the current position x i In the attention calculation process, through the definition of this local neighborhood, the model can capture the local variation relationship of geological features in space, which is consistent with the characteristic that geological phenomena are associated within a certain spatial range.

[0038] Boundary processing: Since geological data may have special situations at the boundaries, such as the neighborhood near the boundary of the study area may not be completely within the data range, the boundary processing operation is to make reasonable supplements or adjustments to these special situations to ensure the accuracy and completeness of the attention calculation and avoid information loss or miscalculation due to boundary problems.

[0039] Calculate x i And the QK scores of all positions in the neighborhood: In the attention mechanism, Q (query), K (key), and V (value) are the core calculation elements. Here we calculate x i The QK score of the location in the neighborhood is used to measure the degree of correlation between the current location and the neighborhood location. This score will serve as an important basis for subsequent weight allocation, reflecting the difference in the importance of different locations on the current location.

[0040] Softmax normalization: Perform Softmax normalization on the calculated QK score so that all neighboring positions are equal to the current position x. i The sum of the weights is 1, and the relevance scores are converted into probability distribution form for subsequent weighted summation to ensure that the weights are reasonable and standardized in the process of attention allocation.

[0041] The weighted summation is the output y i :According to the weights normalized by Softmax, the V values ​​corresponding to each position in the neighborhood are weighted summed to obtain the current position x i The output y i This output integrates the information of neighboring locations and is reasonably weighted according to the correlation, so that the final feature representation of each location not only contains its own information, but also incorporates the key information of local related locations, effectively capturing the local characteristics of geological space.

[0042] Output sequence: After the above series of operations, the output sequence Y=y1,y2,…,y nThis sequence is the geological position feature sequence processed by the sliding window attention mechanism. Compared with the input sequence, it better reflects the local correlation of geological space and provides more geologically meaningful feature data for subsequent geological model construction.

[0043] In the above sliding window processing process, in order to reflect the spatial correlation, it is also necessary to introduce a relative position bias based on the relative position of the coordinates in the window. The calculation method is as follows: Figure 4 By calculating the coordinate distance within the neighborhood, a bias term is generated. When calculating the QK score, introducing the bias term can enhance spatial correlation.

[0044] In addition, since geological features are usually directional, in order to improve the deep integration of the multi-head attention mechanism and geological characteristics, the azimuth between positions can be used as an additional attention weight, and independent attention heads can be assigned to different directions. For the specific process, see Figure 5 , including: directional feature extraction: extracting directional parameters such as fault strike and rock layer dip (such as strike angle and dip) from geological data; coordinate embedding: using NeRF encoding to convert three-dimensional coordinates into high-dimensional features; direction encoding: converting directional parameters into sine / cosine direction vectors; anisotropic kernel initialization: initializing independent attention heads according to the direction of the geological structure (X, Y, Z axis), with each head corresponding to feature weights in different directions; direction weight matrix generation: combining coordinate embedding and direction encoding to calculate the azimuth between positions and generate a weight matrix reflecting the directional correlation of geological structures. Attention fusion: combining the direction weight matrix with the conventional attention mechanism to output an attention sequence containing directional features for geological body prediction.

[0045] S3. Geological feature fusion based on multi-scale convolution and fault constraints: In this step, a convolutional neural network is used to perform multi-scale convolution on the input geological map to extract multi-scale feature data. The fault lines represented by vectors are traversed, and the orientation relationship is calculated based on the spatial distance and angle between the three-dimensional geological coordinates and the fault lines to generate a cross-fault weight mask matrix reflecting the fault constraints. The multi-scale feature data, the attention feature sequence, and the cross-fault mask matrix are fused to obtain comprehensive features.

[0046] The multi-scale convolutional attention calculation process uses geological maps as raw input data. This data covers geological structural information at different scales, from subtle joint textures to macroscopic faults and strata distribution. It provides a basic data source for subsequent multi-scale analysis and reflects the true complex structure of the geological body. The specific process includes: Patch Embedding (16x16 convolution): First, a 16x16 convolution operation, or PatchEmbedding, is performed on the input. This divides the original image into multiple 16x16 patches. Convolution then extracts the features of each patch, converting the image data into a feature sequence. This preliminarily integrates local information while reducing the data dimension for subsequent processing. The features of different patches contain geological details of different local areas, laying the foundation for multi-scale analysis.

[0047] Scale 1: 1 / 4 resolution of original image, multi-head attention (window size 8x8), MaxPool stride = 2: downsample the feature sequence after Patch Embedding to obtain the original Figure 1 A feature map with a resolution of 1 / 4 is used. At this resolution, a multi-head attention mechanism with a window size of 8x8 is used. The multi-head attention mechanism captures features from different subspaces through multiple parallel attention heads to enhance feature expression capabilities. At the same time, a maximum pooling (MaxPool) operation with a stride of 2 is combined to further reduce the resolution and extract the local feature L1 at this scale. This scale focuses on a relatively small range but still includes a certain area of ​​geological structure, such as a meter-level joint network, to mine local detailed information.

[0048] Scale 2: 1 / 8 original image resolution, multi-head attention (window size 4x4), MaxPool stride = 2: Downsampled again to 1 / 8 original image resolution, using a multi-head attention mechanism with a window size of 4x4 and a MaxPool operation with the same stride of 2. This captures geological features at a larger scale and a more macroscopic level than at scale 1, obtaining the local feature L2. This helps integrate geological information at a slightly larger scale, such as rock mass structures at a scale of tens of meters, gradually transitioning from fine-grained to coarse-grained, and constructing a multi-scale feature system.

[0049] Scale 3: 1 / 16 resolution of original image, multi-head attention (global attention), global feature L3: further downsampled to original Figure 1 At / 16 resolution, a global attention mechanism is used to allow each position to pay attention to all positions in the entire feature map, obtaining the most macroscopic global feature L3, which is used to characterize geological structures at the hundred-meter level or even larger, such as large fault systems, and fully grasp the overall geological pattern.

[0050] Upsampling x2, feature fusion L3+L2: The global feature L3 is upsampled (amplified twice) to increase its resolution, and then fused with the local feature L2 at scale 2. By fusing features at different scales and integrating microscopic and macroscopic geological information, the model can understand both the local fine structure and the overall layout. For example, it combines joint networks with rock structure information to provide comprehensive and multi-level feature support for geological knowledge.

[0051] 1x1 convolution adjusts channels: Since the number of channels may change after feature fusion or may not meet the requirements of subsequent processing, 1x1 convolution is used to adjust the channels of the fused features. The number of channels can be flexibly changed without changing the resolution of the feature map, ensuring that the data dimension is adapted to subsequent operations and optimizing feature expression.

[0052] Upsampling x2, feature fusion L2+L1: The features after channel adjustment in the previous step are upsampled again (amplified twice) and fused with the local features L1 at scale 1. This further integrates geological information from the finest to the coarser scales, completing a deep fusion of multi-scale information. The final output features combine geological details at all levels with the macrostructure.

[0053] On the basis of the above multi-scale feature extraction, in order to integrate geological prior knowledge into the attention mechanism and ensure that the output of the model conforms to the geophysical laws, this scheme uses fault impenetrability as an attention mask. The generation process is shown in Figure 6 , including the following steps: Create an all-ones mask matrix: Initialize an N×N all-ones matrix as the initial state of the mask, where N corresponds to the dimension of the geological location data. This mask will be adjusted according to geological rules in the subsequent process to block location information that does not meet geophysical constraints, ensuring that the model follows geological principles for feature learning.

[0054] Traversing fault lines: Traverse the fault line information in the geological data. Faults have special properties in geological structures, such as physical constraints such as impenetrability, and are the focus of attention. By traversing, determine which locations are related to faults so that the mask matrix can be modified in a targeted manner.

[0055] No (fault lines not traversed): During the traversal process, if all fault lines have not been checked, the program continues to perform fault line traversal and subsequent related operations to ensure that the physical constraints brought by all faults are fully considered and no key geological information is missed.

[0056] Calculate the orientation of each coordinate relative to the fault: For each geological position coordinate associated with the fault, calculate its orientation relative to the fault. Orientation information is crucial for determining how geological features change near the fault. For example, orientation can be used to determine whether a location is on the hanging wall or footwall of the fault. Different orientations are affected differently by the fault, providing a basis for subsequent accurate mask setting.

[0057] Return the mask matrix ( side = computesideoffault(coords,fault) ): Generate a cross-fault mask based on the calculated results of the coordinates' orientation relative to the fault. Modify the initial all-one mask matrix by setting the mask values ​​to 0 at locations that do not conform to geological rules (such as the specific physical relationship between the two sides of the fault). This shields the influence of these locations in the attention calculation, forcing the model to follow geophysical constraints such as fault impenetrability during learning, ensuring the rationality of the geological model.

[0058] The process ends when all fault lines are traversed and the mask matrix is ​​updated. The mask matrix obtained at this time will be used in subsequent attention calculations to ensure that the geological model construction process fully considers physical constraints and outputs results that conform to the actual geological situation.

[0059] S4. Model training and parameter optimization: In this step, the model uses the comprehensive features as input, calculates the loss using a joint loss function that incorporates prediction loss and a fault impenetrability constraint. The model parameters are updated using a backpropagation algorithm, completing the training of the geological prediction model. This joint loss function achieves a coordinated optimization of data fitting and physical laws, enabling the model to learn data features while also adhering to geological priors, improving its generalization and predictive reliability in complex geological scenarios.

[0060] S5. Attention visualization and model verification: In this step, the geological data is input into the trained geological body prediction model, and the attention mechanism within the model assigns attention weights to elements at different positions in the input data. These weights constitute the attention weight matrix; the obtained attention weight matrix is ​​then normalized using the softmax function; according to the normalized attention weight matrix, each value in the matrix is ​​mapped to the corresponding color in the selected color mapping scheme; then, Python's drawing library is used for drawing to obtain a visual attention heat map; finally, the data in the heat map is aligned with the data in the three-dimensional geological model, and the drawn heat map is superimposed on the three-dimensional geological model in the form of color coding. After superposition, the key areas are highlighted, achieving the geological interpretability of the key areas.

[0061] It should be noted that although the preferred embodiments of the present invention have been described above, it is understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and all of these changes and modifications do not depart from the scope of protection of the present invention.

Claims

1. A multi-head attention model training method integrating geological rules, characterized by: The following steps are involved: S1. The three-dimensional geological coordinate data is converted into a high-dimensional vector by Fourier eigenmapping, and the geological attribute data is encoded into a feature vector by an embedding layer and a linear transformation; the high-dimensional vector and the feature vector are then concatenated and fused to obtain a fused feature vector; S2. Based on the fused feature vector, a sliding window is used for attention processing, while introducing a relative position bias calculated based on the relative position of the coordinates within the window. By extracting the directional characteristics of the geological structure and calculating the directional weight matrix, anisotropic attention heads are assigned according to the directional characteristics of the geological structure, obtaining an attention feature sequence that contains both spatial locality and directional characteristics. S3. Perform multi-scale convolution on the input geological map using a convolutional neural network to extract multi-scale feature data. Traverse the fault lines represented by vectors and calculate the orientation relationship based on the spatial distance and angle between the three-dimensional geological coordinates and the fault lines to generate a cross-fault weight mask matrix that reflects the fault constraints. Fusion the multi-scale feature data, the attention feature sequence, and the cross-fault mask matrix to obtain comprehensive features. S4. Using the comprehensive features as the input of the model, calculating the loss value using a joint loss function including the prediction loss and the fault impenetrability constraint term, updating the model parameters using a backpropagation algorithm, and completing the training of the geological body prediction model; S5. Based on the input geological data, the trained geological body prediction model is used to obtain the attention weight matrix, which is normalized by the softmax function. The heat map is drawn and superimposed with the 3D geological model.

2. The multi-head attention model training method integrating geological rules according to claim 1, characterized in that: In step S1, the geological attribute data includes: lithology, fault, and lithofacies data.

3. The multi-head attention model training method integrating geological rules according to claim 1, characterized in that: In step S2, the attention processing using a sliding window includes: Define the sliding window radius, and for the sequence representation of the fused feature vector, calculate the attention scores between the elements at each position in the sequence and the elements at other positions in its neighborhood. For the element at each position, a weighted sum is performed based on the attention scores of the elements at other positions in its neighborhood and the element value at the corresponding position to obtain the output sequence after attention processing.

4. The multi-head attention model training method integrating geological rules according to claim 3, characterized in that: In step S2, the introduction of the relative position offset calculated based on the relative position of the coordinates within the window refers to: In attention processing using a sliding window, when calculating the attention score between an element at each position in the sequence and elements at other positions in its neighborhood, the position bias calculated by the relative position of the coordinates within the window is taken into account.

5. The multi-head attention model training method integrating geological rules according to claim 1, characterized in that: In step S2, the direction weight matrix is ​​calculated using the direction cosine matrix according to the strike and dip of the geological structure.

6. The multi-head attention model training method integrating geological rules according to claim 1, characterized in that: In step S3, performing multi-scale convolution on the input geological map by using a convolutional neural network to extract multi-scale feature data includes: First, the input geological map is processed by Patch Embedding through 16×16 convolution to obtain a feature sequence; Then process it at three scales in turn: Scale 1 takes 1 / 4 resolution of the original image, and uses 8×8 window multi-head attention combined with a maximum pooling operation with a stride of 2 to extract the local feature L1; At scale 2, the original image is reduced to 1 / 8 of its original resolution, and the local feature L2 is extracted using a 4×4 window of multi-head attention combined with a maximum pooling operation with a stride of 2. Scale 3 takes 1 / 16 resolution of the original image and obtains the global feature L3 through global attention; Next, L3 is upsampled and initially fused with L2. After adjusting the channel through 1×1 convolution, the initial fused features after channel adjustment are upsampled again and fused with L1 to complete the multi-scale feature integration.

7. The multi-head attention model training method integrating geological rules according to claim 1, characterized in that: In step S3, the fault line represented by the traversal vector is traversed, and the orientation relationship is calculated based on the spatial distance and angle between the three-dimensional geological coordinates and the fault line to generate a cross-fault weight mask matrix reflecting the fault constraint, including: Create an all-1 mask matrix: Create an all-1 matrix of dimension N×N, where N is the dimension of geological location data; Traverse fault line data: Traverse the fault lines in the geological map one by one to locate all areas where physical constraints need to be applied; Calculate the coordinate and orientation relationship: for each geological coordinate point, calculate its spatial orientation relative to the currently traversed fault line; Generate cross-fault mask: Based on the orientation calculation results, the positions in the initial mask matrix that do not conform to geological rules are set to 0 to form a cross-fault weight mask matrix that reflects the fault constraints.

8. A multi-head attention model training method integrating geological rules according to any one of claims 1 to 7, characterized in that: In step S5, based on the input geological data, the trained geological body prediction model is used to obtain the attention weight matrix, the attention weight matrix is ​​normalized by the softmax function, and a heat map is drawn and superimposed with the three-dimensional geological model for output, including: The geological data is input into the trained geological volume prediction model. The attention mechanism within the model assigns attention weights to elements at different positions in the input data. These weights constitute the attention weight matrix. The obtained attention weight matrix is ​​normalized using the softmax function; According to the normalized attention weight matrix, each value in the matrix is ​​mapped to the corresponding color in the selected color mapping scheme; Use Python's drawing library to draw and obtain a heat map; Align the data in the thermal map with the data in the 3D geological model; The drawn thermal map is superimposed on the 3D geological model in a color-coded form.

Images