A microseismic P-wave first arrival picking method based on hybrid deep neural network

Abstract

The present invention discloses a method for picking up the first arrival of microseismic P waves based on a hybrid deep neural network, and relates to the technical field of seismic data processing. The method comprises: obtaining gather data containing microseismic events; determining the first arrival picking result of microseismic P waves according to a UNet-Transformer hybrid deep neural network and the gather data; the UNet-Transformer hybrid deep neural network comprises a UNet network and a Transformer head network connected in sequence; the UNet network comprises an encoder and a decoder connected in a U-shaped manner; the Transformer head network comprises a feature embedding layer, a converter and a feedforward network connected in sequence; the decoder is also connected to the feature embedding layer. The present invention can achieve the first arrival picking of P waves by retaining the local and global context information in the neural network and capturing the long-range relationship between the input elements.

Description

Technical Field

[0001] The present invention relates to the technical field of seismic data processing, and in particular to a method for picking the first arrival of microseismic P waves based on a hybrid deep neural network. Background Art

[0002] The microseismic effective signal first arrival picking technology is one of the important technologies for microseismic data processing. The quality of microseismic monitoring directly depends on the accurate positioning of microseismic events. Accurate P-wave arrival time picking is very important for the above work. However, due to the influence of the on-site monitoring environment, the acquisition of microseismic signals will inevitably superimpose various noise signals, so that the original microseismic record containing the entire process of the event appears to be submerged by noise in the time domain, which brings great difficulty to the P-wave arrival time picking work. Traditional microseismic P-wave picking technology not only cannot meet the requirements of effective signal extraction accuracy under weak signal-to-noise ratio, but also often consumes a lot of time, which obviously cannot meet the needs of today's big data era. However, this process is expensive, labor-intensive, and usually requires rich experience. In contrast, deep learning-based models have shown excellent performance and can automatically segment targets of interest with high accuracy and speed because they can learn and understand complex patterns and features in medical images. Therefore, automated medical image segmentation based on deep learning is in high demand and favored in clinical practice.

[0003] As an important branch of artificial intelligence, deep learning has received more and more attention in recent years. Its characteristics of multiple parameters and large capacity enable deep networks to have powerful processing capabilities for massive data. However, traditional deep neural networks cannot accurately learn the fine features of local detail signals in small sample data sets, resulting in a significant decrease in detection accuracy.

[0004] Among them, convolutional neural networks (CNNs) have shown extremely high results and great potential in many medical image segmentation tasks, especially UNet, a fully convolutional network consisting of a symmetrical encoder and decoder structure with skip connections to pass features from the encoder path to the decoder path. However, since these architectures are usually unable to capture long-range dependencies and global contextual information in images, they usually have poor segmentation performance on target information that varies significantly between patients in terms of texture, shape, and size, and cannot achieve efficient P-wave arrival time picking. Summary of the invention

[0005] The purpose of the present invention is to provide a microseismic P-wave first-arrival picking method based on a hybrid deep neural network, which can realize P-wave first-arrival picking by retaining local and global context information in the neural network and capturing the long-range relationship between input elements.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A microseismic P-wave first arrival picking method based on a hybrid deep neural network, comprising:

[0008] Obtain gather data containing microseismic events;

[0009] According to the UNet-Transformer hybrid deep neural network and the gather data, the microseismic P wave first arrival picking result is determined; the UNet-Transformer hybrid deep neural network includes a UNet network and a Transformer head network connected in sequence; the UNet network includes a U-shaped connected encoder and a decoder; the Transformer head network includes a feature embedding layer, a converter and a feedforward network connected in sequence; the decoder is also connected to the feature embedding layer;

[0010] Among them, the UNet network is used to preprocess the track data to obtain bridge layer data; the preprocessing includes maximum pooling, jump connection and upsampling; the Transformer head network is used to predict the bridge layer data to obtain the microseismic P wave first arrival picking result.

[0011] Optionally, the training process of the UNet-Transformer hybrid deep neural network is:

[0012] Acquire the track data for training and divide it into a training set and a validation set; the training set includes the track training data and the corresponding label data; the label data is the first arrival function on the track picked by manual annotation; the data volume of the label data accounts for 20%-30% of the total track data volume;

[0013] Constructing an initial training model of the UNet-Transformer hybrid deep neural network;

[0014] The initial training model is trained using the training set, and the training is performed with the goal of minimizing the loss between the P wave first arrival picking prediction result output by the initial training model and the label, to obtain a trained UNet-Transformer hybrid deep neural network;

[0015] The verification set is input into the trained UNet-Transformer hybrid deep neural network, and the network is allowed to complete semantic segmentation processing of two adjacent parts of the initial boundary on the gather data, and automatically pick the first arrival according to the segmentation result.

[0016] Optionally, the processing process of the feature embedding layer is:

[0017] Set the size of the bridge layer with the bridge channel to (H, W, C b ), where H, W and C b They represent the height, width, and number of bridged channels, respectively. They are merged using a convolutional layer with a kernel size of E, a stride of S, and padding of P of 3, 4, and 1, respectively. After convolution, the output resolution of the bridge layer is calculated as:

[0018]

[0019]

[0020] In the formula, H out Indicates the height output data, W out Indicates width output data;

[0021] Based on the output resolution of the bridge layer, the process of merging and embedding features into the sequence is expressed as:

[0022]

[0023] In the formula, F f represents the embedding feature, F b Indicates the bridge layer.

[0024] Optionally, the processing process of multi-head attention in the converter is:

[0025] The input data is divided into multiple heads through the multi-head attention mechanism, and the dimension of each head h is d h , set the length of the embedding vector d N =64, the number of heads is h=4, and the attention in each head is calculated as:

[0026]

[0027] In the formula, N represents the number of input sequences; R represents the reduction rate;

[0028] Merge the attention of each head and get the final attention matrix:

[0029] MultiHead(Q,K,V)=Concat(head1,...,head h )W O ,

[0030]

[0031] In the formula, represents the parameter matrix on each head, is the total parameter matrix.

[0032] Optionally, the processing process of the feedforward network is:

[0033] Receive the embedded sequence output by the converter, extract features and upsample using a bilinear interpolation function, obtain the upsampled features, input the upsampled features into the CBR block for further processing, and finally generate the final prediction result; the CBR block consists of 3 convolutional layers, where the convolution kernel size E of the first two layers is 3×3, followed by batch normalization and ReLU activation, and the convolution kernel size E of the third layer is 1×1.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The present invention discloses a microseismic P wave first arrival picking method based on a hybrid deep neural network, the method comprising obtaining gather data containing microseismic events; determining the microseismic P wave first arrival picking result according to a UNet-Transformer hybrid deep neural network and the gather data; the UNet-Transformer hybrid deep neural network comprises a UNet network and a Transformer head network connected in sequence; the UNet network comprises an encoder and a decoder connected in a U-shaped manner; the Transformer head network comprises a feature embedding layer, a converter and a feedforward network connected in sequence; the decoder is also connected to the feature embedding layer. The present invention can realize P wave first arrival picking by retaining local and global context information in the neural network and capturing the long-range relationship between input elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 It is a schematic diagram of the network structure and flow of the microseismic P-wave first arrival picking method based on a hybrid deep neural network of the present invention;

[0038] Figure 2 This is the UNet network structure diagram in this embodiment;

[0039] Figure 3 This is a structural diagram of the decoder in this embodiment;

[0040] Figure 4 This is a diagram of the internal structure of the Transformer head in this embodiment;

[0041] Figure 5This is a schematic diagram of converting typical microseismic event waveform data into P-wave initial picking results using network output segmentation results in this embodiment. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The purpose of the present invention is to provide a microseismic P-wave first-arrival picking method based on a hybrid deep neural network, which can realize P-wave first-arrival picking by retaining local and global context information in the neural network and capturing the long-range relationship between input elements.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] like Figure 1 As shown, the present invention provides a microseismic P wave first arrival picking method based on a hybrid deep neural network, comprising:

[0046] Step 100: Acquire gather data containing microseismic events;

[0047] Step 200: Determine the microseismic P wave first arrival picking result according to the UNet-Transformer hybrid deep neural network and the gather data; the UNet-Transformer hybrid deep neural network includes a UNet network and a Transformer head network connected in sequence; the UNet network includes a U-shaped connected encoder and a decoder; the Transformer head network includes a feature embedding layer, a converter and a feedforward network connected in sequence; the decoder is also connected to the feature embedding layer;

[0048] Among them, the UNet network is used to preprocess the track data to obtain bridge layer data; the preprocessing includes maximum pooling, jump connection and upsampling; the Transformer head network is used to predict the bridge layer data to obtain the microseismic P wave first arrival picking result.

[0049] Based on the above technical solution, the following embodiments are provided.

[0050] To address the shortcomings of the background technology, the current research proposes to implement a self-attention mechanism based on the properties of CNN. It is worth noting that the Transformer was originally conceived for sequence-to-sequence tasks in the NLP framework. It has emerged as an alternative architecture that completely abandons the convolution operator and relies entirely on the attention mechanism, and has a huge impact in the computer vision (CV) community.

[0051] Unlike previous CNN-based methods, Transformer is not only good at capturing global contextual information, but also shows strong adaptability to downstream tasks when pre-trained on a large scale. For example, the first visual Transformer (ViTs) based entirely on self-attention for image recognition debuted in ViT and achieved competitive results on ImageNet after pre-training on a wide range of external datasets using 2D image patches with position embeddings as input sequences. DETR adopts a comprehensive end-to-end object detection method based on Transformer, deeply exploring the connection between objects and the relationship between the overall image background for object detection. SETR replaces the traditional encoder in the standard encoder-decoder network with Transformer, effectively achieving the latest results in natural image segmentation tasks.

[0052] Although Transformer is good at capturing global context, it has difficulty grasping fine details, especially in medical images. To overcome this limitation, researchers have worked hard to integrate CNN-based and Transformer-based models with each other. In particular, TransUNet and TransFuse are representative methods that combine Transformer and UNet and were used in medical image segmentation in the early days.

[0053] As part of the ongoing efforts to leverage the strengths of CNN and Transformer-based models, this paper uses a simple yet effective UNet-Transformer model, named UNetTrans, to achieve fine segmentation of seismic data images, thereby achieving the task of first arrival picking of seismic waves. In this method, the UNet model is designed as a feature extractor to extract multiple feature maps from the input image, which are then input into a bridge layer that is introduced to connect the UNet and Transformer in sequence. At this stage, pixel-level embedding techniques are used instead of positional embedding vectors to make the model more efficient.

[0054] In addition, the Transformer head plays a central role in modeling the relationships and dependencies between input sequences, ultimately generating predictions of the input image and thus achieving P-wave picking. By leveraging the UNet architecture and the Transformer mechanism, this model not only preserves the local and global contextual information, but also captures the long-range relationships between input elements.

[0055] UNet-Transformer network method principle

[0056] While Transformer is good at capturing global information, it has difficulty grasping fine details, especially in seismic data images. To overcome this limitation, a CNN-based and Transformer-based model was integrated with each other. The model uses UNet as its backbone and includes a Transformer connector. Basically, the role of the backbone is to receive the input image and then output bridge layers, which are processed by the Transformer connector to obtain the final prediction results. The backbone architecture is characterized by the U-shaped structure of the Unet network, which includes an encoder part and a decoder part. Generally, the overall architecture of the integrated UNet-Transformer is as follows Figure 1 shown.

[0057] The following are the details of the network structure:

[0058] (1) Encoder

[0059] The encoder part is responsible for extracting features from the input image in the network. Usually, it consists of several convolutional layers, called UNet blocks, followed by a max pooling layer. These UNet blocks gradually reduce the spatial size of the input image while increasing the depth (number of channels) of the feature map. The network used here constructs an encoder part containing 4 UNet blocks. The specific construction of the UNet block is as follows Figure 2 shown.

[0060] The UNet block includes two convolutional neural networks (Conv), followed by a batch normalization function (BatchNorm) and a rectified linear unit (ReLU) activation function. The structure of the UNet block can be expressed as:

[0061]

[0062] in, and F i are the intermediate and final features of each UNet block, respectively. C in , C h , C o They are the input, hidden and output layers respectively.

[0063] (2) Decoder

[0064] The decoder part is responsible for upsampling the feature image to the original image size and generating the bridge layer. Typically, the decoder includes 4 decoder blocks, each of which includes upsampling (or deconvolution), skip connection, and UNet blocks.

[0065] First, the decoder upsamples the size of the previous layer, then concatenates them with the corresponding layer in the encoder using skip connections, and finally passes the concatenated features to the UNet block. Skip connections allow the network to merge low-level and high-level features, providing more information about the features.

[0066] Typically, the last layer of the decoder usually has a single channel (for binary segmentation) or multiple channels (for multi-class segmentation), where each channel represents the probability of a pixel belonging to a specific class. However, here, instead of extracting a segmentation map, another convolutional layer is applied to the output of the decoder to obtain a bridge layer. These layers will be used as input to the Transformer head.

[0067] (3) Transformer Head

[0068] The Transformer head first merges the features of the bridge layer through a convolutional layer. Subsequently, these merged features are flattened into a sequence and passed into the multi-head attention (MHA) mechanism. The output of the MHA is passed to a multi-layer perceptron (MLP), which is mainly used to map the input features to the output features. Finally, the output from the MLP is linearly upsampled and processed through a convolutional layer in the CBR block before outputting the final prediction. The structure of the Transformer head is shown in Figure 2. Figure 4 shown.

[0069] 1) Feature Embedding:

[0070] The size of the bridge layer with bridge channels is (H, W, C b ), where H, W and C b They represent the height, width, and number of bridged channels, respectively, and are merged using a convolutional layer with a kernel size of E, a stride of S, and padding of P of 3, 4, and 1, respectively. After convolution, the output resolution of the bridge layer is calculated as:

[0071]

[0072] In the context of image segmentation, the goal is to establish the relationship between pixels in an image. This can be achieved through various methods, such as CNN-based techniques, attention mechanisms, and graph neural networks. For this particular study, the inventors utilized the attention mechanism because it is very effective in capturing long-range features. In the proposed method, each pixel and its variations in different spatial dimensions (represented by various features in different channels) are treated as a single input vector, denoted as α. In other words, the merged features are flattened into a sequence with the dimension of Where N = H out ×W out .

[0073] Unlike the Transformer approach in computer vision, this approach does not use positional embedding vectors when flattening the input images. This decision stems from the fact that the input images are merged and the merged features are embedded at the pixel level. In general, the process of merging and embedding the merged features into a sequence can be expressed as:

[0074]

[0075] 2) Converter:

[0076] The transformer (also called Transformer block) basically includes multi-head attention, multi-layer perceptron, LayerNorm and residual connection. The Transformer block can be represented as:

[0077]

[0078] again, and F i are the intermediate layer and output layer of the i-th layer of the Transformer block. For the first Transformer block or when i=1, the input data is the embedded feature (F f ).

[0079] In Multi-Head Attention (MHA), the dependencies between a sequence and other sequences are calculated by using cross attention. In this step, the computational complexity is N 2 , where N is the number of input sequences. To reduce the computational effort, a sequence reduction technique is used here to make it applicable to high-resolution input images. Therefore, the complexity becomes N 2 / R, where R is the reduction rate.

[0080] The input sequence is divided into multiple heads, and the dimension of each head h in MHA is d h In this research method, the length d of the embedding vector is used N = 64, and the number of heads is h = 4. The attention in each head is calculated as follows:

[0081]

[0082] In the formula,

[0083] Once the attention of each head is calculated, the authors merge them all together to get the final attention matrix:

[0084]

[0085] In the formula, represents the parameter matrix on each head, is the total parameter matrix.

[0086] The output of the MHA is added to its input through a residual connection. This connection enables the network to learn residual information, or the difference between the desired output and the current prediction. By doing so, the network can more easily capture and propagate gradients during training, even for very deep networks. This helps to efficiently train deeper neural networks and alleviates the gradient vanishing problem. In addition to the MHA, the Transformer block also includes a connected feedforward network or MLP, which consists of two linear transformations and a GeLU activation in the middle. In particular, the merged features are normalized and then fed to the MLP. Similar to the output from the MHA, another residual connection is used here to add the output of the MLP to its input. Equation 4 describes the process of MHA and MLP, where the input features are mapped to the output features as per the standard Transformer. The process of the Transformer block can be repeated multiple times D, and in the method, D=3 is used.

[0087] 3) Feedforward Network:

[0088] The feed-forward network (FFN) receives the embedded sequence from the Transformer block to extract features and generate prediction maps. Since the FFN operates on sequences as input, these inputs need to be reshaped into the required input shape (H out ,W out ,C b ).

[0089] In addition, the input shape undergoes a merge operation, resulting in a 4-fold reduction in size. Therefore, the reshaped features must be upsampled 4 times to match the original input shape (H, W, C b ). This upsampling process uses a bilinear interpolation function to increase the resolution of the feature map.

[0090] In mathematical formula, this step can be expressed as:

[0091] F rs=Upscale(Reshape(F D )) (7)

[0092] Here, F rs represents the reshaped upsampled feature map, F D are the features from Transformer block D, the final Transformer block.

[0093] After obtaining the upsampled features, they are fed to the CBR block for further processing to ultimately generate the final predictions. Named for its convolutional layers, batch normalization, and ReLU activations, the CBR block plays a vital role in the refinement and spatial enhancement of features, enabling the network to capture complex patterns and relationships in the data. CBR consists of 3 convolutional layers, where the first two layers have a convolution kernel size E of 3×3, followed by batch normalization and ReLU activation, while the third layer has a convolution kernel size E of 1×1, accepts features from the previous layer and directly outputs the final prediction M. Mathematically, this can be expressed as:

[0094]

[0095] again, is the intermediate output of the CBR block. h1 and C h2 They are the hidden layers of the first and second convolutional layers, respectively. In the present invention, a SEMI-Net model for seismic data image segmentation is constructed, and the final prediction M is a binary image, and then the binary recognition result is converted into the seismic wave first arrival.

[0096] The following specific experiments were carried out according to the above specific structure.

[0097] (1) Data preprocessing

[0098] In this case, the preprocessing process is to generate the original microseismic gather data generated by the long and short time window method into the training samples that are finally input into the hybrid neural network model.

[0099] In the preprocessing step, this embodiment needs to convert the seismic trace data in the training set into grayscale images to facilitate subsequent deep learning processing. Figure 5In the example, due to the event track data. In the preprocessing, this embodiment needs to mark the first arrival of the track grayscale image. Due to the complexity of the first arrival function in the actual data, this embodiment only selects to mark those obvious and certain first arrival functions when marking the first arrival function, and ignores those relatively difficult to define parts. This can make the positive samples input into the hybrid neural network (referring to the first arrival function part) as true as possible. It is worth noting that if the prediction of the hybrid neural network can identify those relatively small first arrival functions in the processing results, it can verify the effectiveness of the hybrid network in extracting the first arrival function features in this method from the side.

[0100] In the preprocessing, in order to highlight the first arrival function area, this embodiment needs to crop the grayscale image and label of the original section to form a new grayscale image and new label in the first arrival function set. The image cropping range selected in this process is mainly based on the first arrival function distribution in the original label image, and the size of the generated new label is 651×830. After obtaining the new label, the same cropping operation is performed on the original grayscale image to generate a new grayscale image in the corresponding first arrival function set, and the size of the new grayscale image is also 651×830.

[0101] This embodiment randomly crops and generates multiple 256×256 samples and corresponding labels based on the newly obtained grayscale image and label. This embodiment performs data enhancement on the samples. Specifically, a part of all samples generated by the 7 original sections is randomly selected for horizontal mirroring, and then the newly generated image is used to replace the image before the transformation as the new sample, so as to keep the number of samples the same as before data enhancement. All the samples finally obtained are used as training samples for input into the convolutional neural network in the next step. The same operation is performed for the labels corresponding to the samples.

[0102] (2) Network training process

[0103] When training the convolutional neural network, this embodiment uses the currently more common NVIDIA Tesla GPU (32GB) for training.

[0104] For the present invention, the number of training samples generated from 200 original gather data is selectable. In addition, the structure of the convolutional network model used in the model is also variable. Due to the network characteristics of UNET, the network depth can be adaptively adjusted according to the data samples.

[0105] After experiments, the training time of the UNet-Transformer network model is about 45 minutes. Considering that the network can make multiple predictions after one training, this training time is completely acceptable.

[0106] (2) Prediction results

[0107] After the neural network is trained, this embodiment uses the network to predict samples in the test set. When predicting an image with a size of 651×830, the prediction time required by the neural network is less than 1 second. Figure 5 This is the effect diagram of the application of this method. The prediction results directly output by the hybrid neural network are converted to generate the automatic picking results of the first arrival of the P wave. From the figure, we can intuitively see the first arrival function position results predicted by the neural network.

[0108] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0109] This article uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A microseismic P-wave first arrival picking method based on a hybrid deep neural network, characterized in that: include: Obtain gather data containing microseismic events; According to the UNet-Transformer hybrid deep neural network and the gather data, the microseismic P wave first arrival picking result is determined; the UNet-Transformer hybrid deep neural network includes a UNet network and a Transformer head network; the UNet network includes a U-shaped connected encoder and decoder; the Transformer head network includes a feature embedding layer, a converter and a feedforward network connected in sequence; the decoder and the feature embedding layer are also connected through a bridge layer; The UNet network is used to preprocess the gather data to obtain bridge layer data; the preprocessing includes maximum pooling, skip connection and upsampling; The Transformer head network is used to predict the bridge layer data to obtain the microseismic P wave first arrival picking result.

2. The microseismic P wave first arrival picking method based on hybrid deep neural network according to claim 1 is characterized in that: Before applying the UNet-Transformer hybrid deep neural network, the method further includes: training the UNet-Transformer hybrid deep neural network; the training process is: Acquire the track gather data for training and divide it into a training set and a validation set; the training set includes the track gather training data and corresponding label data; the label data is the first arrival function on the track gather picked up by manual annotation; Constructing an initial training model of the UNet-Transformer hybrid deep neural network; The initial training model is trained using the training set, and the training is performed with the goal of minimizing the loss between the P wave first arrival picking prediction result output by the initial training model and the label, to obtain a trained UNet-Transformer hybrid deep neural network; The verification set is input into the trained UNet-Transformer hybrid deep neural network, and the network is allowed to complete semantic segmentation processing of two adjacent parts of the initial boundary on the gather data, and automatically pick the first arrival according to the segmentation result.

3. The microseismic P wave first arrival picking method based on hybrid deep neural network according to claim 1 is characterized in that: The processing process of the feature embedding layer is: Set the size of the bridge layer with the bridge channel to (H, W, C b ), where H, W and C b They represent the height, width, and number of bridged channels, respectively. They are merged using a convolutional layer with a kernel size of E, a stride of S, and padding of P of 3, 4, and 1, respectively. After convolution, the output resolution of the bridge layer is calculated as: In the formula, H out Indicates the height output data, W out Indicates width output data; Based on the output resolution of the bridge layer, the process of merging and embedding features into the sequence is expressed as: In the formula, F f represents the embedding feature, F b Indicates the bridge layer.

4. The microseismic P wave first arrival picking method based on hybrid deep neural network according to claim 1 is characterized in that: The converter includes multi-head attention; the processing process of the multi-head attention is: The input data is divided into multiple heads through the multi-head attention mechanism, and the dimension of each head h is d h , set the length of the embedding vector d N =64, the number of heads is h=4, and the attention in each head is calculated as: In the formula, N represents the number of input sequences; R represents the reduction rate; Merge the attention of each head and get the final attention matrix: MultiHead(Q,K,V)=Concat(head1,...,head h )W O , In the formula, represents the parameter matrix on each head, is the total parameter matrix.

5. The microseismic P wave first arrival picking method based on hybrid deep neural network according to claim 1 is characterized in that: The processing process of the feedforward network is: Receive the embedded sequence output by the converter, extract features and upsample using a bilinear interpolation function, obtain the upsampled features, input the upsampled features into the CBR block for further processing, and finally generate the final prediction result; the CBR block consists of 3 convolutional layers, where the convolution kernel size E of the first two layers is 3×3, followed by batch normalization and ReLU activation, and the convolution kernel size E of the third layer is 1×1.

Images