Seismic velocity modeling method and system based on transformer

Abstract

This invention discloses a Transformer-based seismic velocity modeling method and system, applied in the field of artificial intelligence technology. The method includes: obtaining feature vectors; generating an input sequence from the feature vectors; extracting a deep feature sequence containing global wavefield information from the input sequence; rearranging the sequence into a two-dimensional or three-dimensional spatial grid to form a feature image and a predicted velocity model; constructing a hybrid loss function including data fitting loss, physical consistency loss, and model prior loss; and jointly training and optimizing the Transformer encoder layer and the convolutional neural network decoder using the hybrid loss function until the loss converges to obtain the final velocity model. This invention effectively integrates seismic wave spatial information into the analysis model, enabling direct prediction of underground velocities from seismic observation data and significantly improving the spatial continuity of the velocity model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence modeling technology, specifically to a method and system for earthquake velocity modeling based on Transformer. Background Technology

[0002] Seismic velocity modeling is a core component of seismic exploration data processing and interpretation, as it reveals the spatial distribution of seismic wave propagation velocities within the subsurface medium. High-precision velocity models form the basis for pre-stack depth migration imaging, time-depth conversion, reservoir prediction, and geological structure interpretation, ultimately determining the final imaging quality and exploration success rate.

[0003] Currently, underground velocity modeling methods can be mainly divided into the following three categories: 1. Ray tomography inversion: relies on manual picking of travel time and initial model, has uneven ray coverage in complex structural regions, strong nonlinearity, is prone to getting trapped in local extrema, and has a long calculation cycle.

[0004] 2. Full waveform inversion: extremely high computational cost, extremely sensitive to the initial model, and has the problem of periodic jumps.

[0005] 3. Convolutional Neural Network Methods: Local receptive field: difficult to capture long-range dependencies in seismic data; insensitive to sequences: seismic data is a spatiotemporal sequence, and CNNs have weak time-series processing capabilities.

[0006] In the prior art, Chinese patent application number 202511952189.9 discloses a multimodal joint inversion imaging method for transient electromagnetic and seismic waves based on physical information Transformer, belonging to the interdisciplinary field of geophysical exploration and artificial intelligence. It includes: preprocessing the raw transient electromagnetic and seismic wave data by anomaly detection, interpolation, filtering, and normalization; extracting features characterizing electrical properties, elasticity, and cross-physical significance; serializing spatial data through a gridding and alternating fusion strategy, and constructing an enhanced code that integrates absolute and relative positions and physical attributes; designing a physically constrained encoder-decoder architecture for multi-scale forward and inversion modeling; and quantifying the uncertainty of the inversion results using the Bayesian Monte Carlo method to generate a confidence map. However, it lacks analysis of the spatial distribution of seismic wave propagation velocity, does not fully and accurately utilize spatial information, and is not conducive to analyzing the distribution of seismic waves in the subsurface medium. Summary of the Invention

[0007] In order to at least overcome the above-mentioned shortcomings in the prior art, the purpose of this application is to provide a Transformer-based seismic velocity modeling method and system.

[0008] In a first aspect, embodiments of this application provide a Transformer-based seismic velocity modeling method, including: Raw seismic data is acquired and preprocessed to extract multidimensional geophysical attributes from multiple sampling locations, forming multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows. The feature vector is mapped to a high-dimensional feature embedding vector through a learnable linear layer, generating a physical location embedding vector corresponding to each sampling position. The high-dimensional feature embedding vector and the corresponding physical location embedding vector are then fused to form an input sequence. The physical location embedding vector includes spatial coordinate information and temporal information of the observation system. The input sequence is fed into a stacked Transformer encoder layer, and a deep feature sequence containing global wavefield information is extracted through a multi-head attention mechanism. The deep feature sequence is rearranged into a two-dimensional or three-dimensional spatial grid to form a feature image based on the corresponding physical location coordinates, and then upsampling and convolution operations are performed by a convolutional neural network decoder with skip connections to form a prediction speed model. A hybrid loss function is constructed, which includes data fitting loss, physical consistency loss, and model prior loss. The Transformer encoder layer and the convolutional neural network decoder are jointly trained and optimized using the hybrid loss function until the loss converges to obtain the final speed model.

[0009] In one possible implementation, the hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

[0010] In one possible implementation, the fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

[0011] In one possible implementation, the generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

[0012] In one possible implementation, the formation of the velocity prediction model includes: The feature image is upsampled and convolutionally processed by a convolutional neural network decoder with skip connections to complete the fusion operation; the skip connections are used to fuse the two-dimensional or three-dimensional encoder feature map obtained by spatial rearranging the output of at least one intermediate layer of the Transformer encoder with the corresponding spatial scale decoder feature map in the convolutional neural network decoder; the generation structure of the input sequence, the generation structure of the deep feature sequence, and the structure of the fusion operation are sequentially concatenated to form the prediction speed model.

[0013] In one possible implementation, the final output of the predicted velocity model is expressed as follows: In the formula, For activation function, To output the projection matrix, The feature map output by the convolutional neural network decoder. For bias terms, The maximum permissible speed value, This is the minimum permissible speed value.

[0014] Secondly, embodiments of this application also provide a Transformer-based seismic velocity modeling system, including: The acquisition unit is configured to acquire raw seismic data, perform preprocessing, and extract multidimensional geophysical attributes from multiple sampling locations to form multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows. The fusion unit is configured to map the feature vector into a high-dimensional feature embedding vector through a learnable linear layer, generate a physical location embedding vector corresponding to each of the sampling locations, and fuse the high-dimensional feature embedding vector and the corresponding physical location embedding vector to form an input sequence; the physical location embedding vector includes spatial coordinate information and temporal information of the observation system; An attention unit is configured to input the input sequence into a stacked Transformer encoder layer and extract a deep feature sequence containing global wavefield information through a multi-head attention mechanism. A convolutional neural network unit is configured to rearrange the deep feature sequence into a two-dimensional or three-dimensional spatial grid to form a feature image based on the corresponding physical location coordinates, and to perform upsampling and convolution operations through a convolutional neural network decoder with skip connections to form a prediction speed model. The training unit is configured to construct a hybrid loss function that includes data fitting loss, physical consistency loss, and model prior loss, and to jointly train and optimize the Transformer encoder layer and the convolutional neural network decoder through the hybrid loss function until the loss converges to obtain the final speed model.

[0015] In one possible implementation, the hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

[0016] In one possible implementation, the fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

[0017] In one possible implementation, the generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

[0018] Compared with the prior art, the present invention has the following advantages and beneficial effects: This invention relates to a Transformer-based seismic velocity modeling method and system that effectively integrates seismic wave spatial information into the analysis model, enabling direct prediction of underground velocities from seismic observation data and significantly improving the spatial continuity of the velocity model. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the method steps in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0021] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Please refer to the following: Figure 1 This is a flowchart illustrating the earthquake velocity modeling method based on Transformer provided in this embodiment of the invention. Further, the earthquake velocity modeling method based on Transformer may specifically include the contents described in steps S1-S5.

[0023] S1: Acquire raw seismic data, perform preprocessing to extract multidimensional geophysical attributes from multiple sampling locations, forming multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows; S2: The feature vector is mapped to a high-dimensional feature embedding vector through a learnable linear layer to generate a physical location embedding vector corresponding to each sampling position, and the high-dimensional feature embedding vector and the corresponding physical location embedding vector are fused to form an input sequence; the physical location embedding vector includes spatial coordinate information and temporal information of the observation system; S3: Input the input sequence into a stacked Transformer encoder layer, and extract a deep feature sequence containing global wave field information through a multi-head attention mechanism; S4: The deep feature sequence is rearranged into a two-dimensional or three-dimensional spatial grid to form a feature image according to the corresponding physical location coordinates, and then upsampling and convolution operations are performed by a convolutional neural network decoder with skip connections to form a prediction speed model. S5: Construct a hybrid loss function that includes data fitting loss, physical consistency loss, and model prior loss, and use the hybrid loss function to jointly train and optimize the Transformer encoder layer and the convolutional neural network decoder until the loss converges to obtain the final speed model.

[0024] In implementing this application, the original seismic data needs to be acquired first. This data can be the original pre-stack seismic gathers or the first arrival travel time data. This data needs to be standardized into a uniform format and then sampled to form feature vectors corresponding to different sampling locations. The sampling method can be data point sampling or sliding window sampling. The multidimensional geophysical properties can include amplitude, instantaneous frequency, instantaneous phase, envelope, relative offset, azimuth, local slope, and local curvature. Then, the feature vector is linearized into a high-dimensional feature embedding vector through a learnable matrix. At this point, physical location also needs to be introduced to form a physical location embedding vector. The two vectors are fused to form the input sequence for subsequent input to the Transformer encoder layer.

[0025] In this embodiment, the fused input sequence is input into a stacked Transformer encoder layer. Each encoder layer can calculate the correlation weights between all elements in the sequence through a self-attention mechanism. This allows the model itself to learn which long-range data should be focused on in order to invert the velocity at a certain point. The output of each attention layer is processed by a feedforward neural network and optimized through residual connections and layer normalization, which can gradually extract deep feature sequences containing global wave field information.

[0026] In this embodiment, the deep feature sequence output by the stacked Transformer encoder layers is one-dimensional data, while the feature image formed by rearranging the corresponding physical location coordinates into a two-dimensional or three-dimensional spatial grid is rich in semantics, thus facilitating processing by the subsequent convolutional neural network decoder. The convolutional neural network decoder, through upsampling and convolution operations, can gradually recover spatial details and ultimately form a complete predicted velocity model. It should be understood that the input data for the predicted velocity model is generally the aforementioned feature vector and its corresponding physical location, and the output data generally includes the velocity data at that point. The overall model is then trained using the raw seismic data obtained above.

[0027] During training, this application constructs a special hybrid loss function, which includes three components: data fitting loss, physical consistency loss, and model prior loss. The data fitting loss measures the difference between the velocity model predicted by the network after forward modeling and the actual observed seismic data. The physical consistency loss ensures that the wave propagation process of the predicted velocity model strictly follows the wave equation or ray theory, which is a strict physical constraint. The model prior loss introduces prior knowledge to constrain the spatial structure and morphology of the velocity model, so that it can better conform to the geological structural characteristics. By repeating the training process until the loss converges, the final usable model is obtained.

[0028] In one possible implementation, the hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

[0029] In the implementation of this application, a more specific hybrid loss function is provided, in which hyperparameter weights are used to balance the importance of the three losses, and are typically set through cross-validation or empirically. In the physical consistency loss, the main network sets the output data of the prediction velocity model. orthogonal data Compare the seismic records of the i-th observation Simultaneously constructing the countermeasure Let be the i-th adversarial data. Far away, but with The noise level should not exceed a preset limit. By introducing a max function, the network prediction model can perform well when interpreting real earthquake data, but perform relatively poorly when interpreting worst-case scenarios (adversarial data) created by human intervention, thus ensuring model robustness and physical consistency. For example, the hyperparameters can be determined through cross-validation: α = 1.0, β = 0.2, γ = 0.02.

[0030] In the embodiments of this application, the geologically sensitive weight function in the prior loss is a predefined or learnable weight based on the spectral characteristics of typical geological structures, such as the scale of sedimentary bedding and the sharpness of salt body boundaries. Traditional total variation regularization only penalizes spatial gradients, which may obscure details, while the prior loss of this application allows for differentiated constraints at different wavenumber-frequency bands. For example, w can be set to have lower weights in the high wavenumber (small scale) part to preserve details such as faults and fractures, and higher weights in the low wavenumber (large scale) part to force background velocity smoothing. This prevents the model from generating unrealistic high-frequency oscillations to fit noise. In the data fitting loss, different loss functions are set for travel-time inversion and full waveform inversion. Travel-time fitting only requires ray tracing or solving equations, and the single calculation time is short, making it suitable for quickly constructing the initial background velocity. Waveform fitting, on the other hand, requires forward modeling of wave equations, which is computationally expensive but has high resolution, making it suitable for the fine inversion stage.

[0031] In one possible implementation, the fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

[0032] When implementing the embodiments of this application, As a learnable linear projection matrix, it can map feature vectors into high-dimensional feature embedding vectors. For location information, this application divides it into depth information and horizontal information. The depth information is embedded through two-way travel time and seismic wave reference velocity, while the horizontal information represents the shot point location and receiver location. The shot point location represents the location of the seismic source, while the receiver location is the horizontal coordinate of the sampling location. The above mechanism can effectively fuse feature vectors and location information. The + in the formula represents the sequential arrangement of vectors to form the final long vector input sequence.

[0033] In one possible implementation, the generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

[0034] In one possible implementation, the formation of the velocity prediction model includes: The feature image is upsampled and convolutionally processed by a convolutional neural network decoder with skip connections to complete the fusion operation; the skip connections are used to fuse the two-dimensional or three-dimensional encoder feature map obtained by spatial rearranging the output of at least one intermediate layer of the Transformer encoder with the corresponding spatial scale decoder feature map in the convolutional neural network decoder; the generation structure of the input sequence, the generation structure of the deep feature sequence, and the structure of the fusion operation are sequentially concatenated to form the prediction speed model.

[0035] In the implementation of this application, the feature image is upsampled and convolutionally fused using a convolutional neural network decoder with skip connections. Unlike existing technologies, the scale features corresponding to the skip connections in this application originate from the Transformer encoder layer; that is, a segment of the skip connection is connected to the Transformer encoder layer, thus artificially constructing multi-scale features. Since the encoder in this application is a Transformer encoder, its output data is one-dimensional and therefore naturally lacks multi-scale information. Therefore, the skip connections in this application can preserve more local waveform details and construction boundary information, injecting the Transformer global context into the local detail recovery process of the convolutional neural network, thereby improving model accuracy. Specifically, the fusion operation includes: The output sequence of at least one intermediate layer of a Transformer encoder layer is rearranged into a two-dimensional or three-dimensional spatial grid according to the corresponding physical location coordinates to form at least two jump feature images. Each of the skip feature images is adapted to match the number of channels, so that the number of channels matches the number of channels in the decoder feature map; The adapted jump feature image and the corresponding decoder feature map are fused together to generate a fused feature map, which is used as the final output feature map of the convolutional neural network decoder.

[0036] In one possible implementation, the final output of the predicted velocity model is expressed as follows: In the formula, For activation function, To output the projection matrix, The feature map output by the convolutional neural network decoder. For bias terms, The maximum permissible speed value, This is the minimum permissible speed value.

[0037] Based on the same inventive concept, this application also provides a Transformer-based seismic velocity modeling system, including: The acquisition unit is configured to acquire raw seismic data, perform preprocessing, and extract multidimensional geophysical attributes from multiple sampling locations to form multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows. The fusion unit is configured to map the feature vector into a high-dimensional feature embedding vector through a learnable linear layer, generate a physical location embedding vector corresponding to each of the sampling locations, and fuse the high-dimensional feature embedding vector and the corresponding physical location embedding vector to form an input sequence; the physical location embedding vector includes spatial coordinate information and temporal information of the observation system; An attention unit is configured to input the input sequence into a stacked Transformer encoder layer and extract a deep feature sequence containing global wavefield information through a multi-head attention mechanism. A convolutional neural network unit is configured to rearrange the deep feature sequence into a two-dimensional or three-dimensional spatial grid to form a feature image based on the corresponding physical location coordinates, and to perform upsampling and convolution operations through a convolutional neural network decoder with skip connections to form a prediction speed model. The training unit is configured to construct a hybrid loss function that includes data fitting loss, physical consistency loss, and model prior loss, and to jointly train and optimize the Transformer encoder layer and the convolutional neural network decoder through the hybrid loss function until the loss converges to obtain the final speed model.

[0038] In one possible implementation, the hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

[0039] In one possible implementation, the fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

[0040] In one possible implementation, the generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

[0041] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0042] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices or units, or may be electrical, mechanical or other forms of connection.

[0043] The units described as separate components may or may not be physically separate. As will be apparent to those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0044] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0045] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0046] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A seismic velocity modeling method based on Transformer, characterized in that, include: Raw seismic data is acquired and preprocessed to extract multidimensional geophysical attributes from multiple sampling locations, forming multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows. The feature vector is mapped to a high-dimensional feature embedding vector through a learnable linear layer, generating a physical location embedding vector corresponding to each sampling position. The high-dimensional feature embedding vector and the corresponding physical location embedding vector are then fused to form an input sequence. The physical location embedding vector includes spatial coordinate information and temporal information of the observation system. The input sequence is fed into a stacked Transformer encoder layer, and a deep feature sequence containing global wavefield information is extracted through a multi-head attention mechanism. The deep feature sequence is rearranged into a two-dimensional or three-dimensional spatial grid to form a feature image based on the corresponding physical location coordinates, and then upsampling and convolution operations are performed by a convolutional neural network decoder with skip connections to form a prediction speed model. A hybrid loss function is constructed, which includes data fitting loss, physical consistency loss, and model prior loss. The Transformer encoder layer and the convolutional neural network decoder are jointly trained and optimized using the hybrid loss function until the loss converges to obtain the final speed model.

2. The seismic velocity modeling method based on Transformer according to claim 1, characterized in that, The hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

3. The seismic velocity modeling method based on Transformer according to claim 1, characterized in that, The fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

4. The seismic velocity modeling method based on Transformer according to claim 1, characterized in that, The generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

5. The seismic velocity modeling method based on Transformer according to claim 1, characterized in that, The formation of the predicted velocity model includes: The feature image is upsampled and convolutionally processed by a convolutional neural network decoder with skip connections to complete the fusion operation; the skip connections are used to fuse the two-dimensional or three-dimensional encoder feature map obtained by spatial rearranging the output of at least one intermediate layer of the Transformer encoder with the corresponding spatial scale decoder feature map in the convolutional neural network decoder; the generation structure of the input sequence, the generation structure of the deep feature sequence, and the structure of the fusion operation are sequentially concatenated to form the prediction speed model.

6. The seismic velocity modeling method based on Transformer according to claim 1, characterized in that, The final output of the predicted velocity model is expressed as follows: In the formula, For activation function, To output the projection matrix, The feature map output by the convolutional neural network decoder. For bias terms, The maximum permissible speed value, This is the minimum permissible speed value.

7. A Transformer-based seismic velocity modeling system, characterized in that, include: The acquisition unit is configured to acquire raw seismic data, perform preprocessing, and extract multidimensional geophysical attributes from multiple sampling locations to form multiple feature vectors containing physical meaning; the sampling locations are seismic data points and / or local time windows. The fusion unit is configured to map the feature vector into a high-dimensional feature embedding vector through a learnable linear layer, generate a physical location embedding vector corresponding to each of the sampling locations, and fuse the high-dimensional feature embedding vector and the corresponding physical location embedding vector to form an input sequence; the physical location embedding vector includes spatial coordinate information and temporal information of the observation system; An attention unit is configured to input the input sequence into a stacked Transformer encoder layer and extract a deep feature sequence containing global wavefield information through a multi-head attention mechanism. A convolutional neural network unit is configured to rearrange the deep feature sequence into a two-dimensional or three-dimensional spatial grid to form a feature image based on the corresponding physical location coordinates, and to perform upsampling and convolution operations through a convolutional neural network decoder with skip connections to form a prediction speed model. The training unit is configured to construct a hybrid loss function that includes data fitting loss, physical consistency loss, and model prior loss, and to jointly train and optimize the Transformer encoder layer and the convolutional neural network decoder through the hybrid loss function until the loss converges to obtain the final speed model.

8. The seismic velocity modeling system based on Transformer according to claim 7, characterized in that, The hybrid loss function is expressed as follows: In the formula, α, β, and γ are hyperparameter weights. For mixed loss, For data fitting loss, For physical consistency loss, For the model's prior loss, Let F be the i-th physical consistency loss value, and let F be the forward modeling operator of the wave equation. For the output data of the predicted velocity model, Let i be the seismic record value of the i-th observation. Let L be the ith adversarial data, L be the representation of the physical residual operator in the spectral domain, k be the spatial wavenumber, ω be the time frequency, and w(k,ω) be the geological sensitivity weight function. The data fitting loss during travel time inversion, The data fitting loss is denoted as N, where N is the total number of travel time data. Let i be the initial arrival time observed. for The i-th predicted travel time is obtained through fast ray tracing or solving the equation of process function. For the i-th The output data of the predicted velocity model at time step. For the i-th The recorded value of the observation at time T is the total number of times.

9. The seismic velocity modeling system based on Transformer according to claim 7, characterized in that, The fusion to form the input sequence includes: The input sequence is formed by fusing according to the following formula: In the formula, Given the input sequence, It is a learnable linear mapping matrix. Let i be the feature vector at the i-th sampling position. The depth vector is embedded in the physical location vector of the i-th sampling position. The vector representing the spatial location is embedded in the physical location vector of the i-th sampling position. Characterization embedding layer, For round-trip travel, The reference velocity for seismic waves is MLP, which stands for Multilayer Perceptron. and Here are the horizontal coordinates of the shot point. and Here are the horizontal coordinates of the receiver point.

10. The seismic velocity modeling system based on Transformer according to claim 7, characterized in that, The generation of deep feature sequences includes: In each of the stacked Transformer encoder layers, the association weights between elements in the input sequence are calculated using a multi-head self-attention mechanism; Each encoder layer obtains the attention output by weighted summation of the sequence using the associated weights; Each encoder layer passes the attention output through residual connections and layer normalization, and then performs a nonlinear transformation through a feedforward network to form the output sequence of this layer. The deep feature sequence is formed by stacking multiple output sequences of this layer layer by layer.

Images