Seismic data output method of seismic processing interpretation system and storage medium

Abstract

The invention provides a seismic data output method of a seismic processing interpretation system, and belongs to the field of seismic exploration. The method comprises the following implementation steps: step 1, establishing a processing flow; 2, acquiring parameters input by a user; step 3, filtering data of the elastic distributed data set RDD according to parameters input by a user; step 4, outputting the channel header temporary file and the channel data temporary file in parallel; 5, extracting file header information of the header file from the elastic distributed data set RDD, and writing the file header information into an independent info file; step 6, merging the trace header temporary files to obtain an hdr file as an output file; merging the trace data temporary files to obtain a data file as an output file; and outputting the data file, the hdr file and the info file. According to the invention, the output efficiency of the seismic data is effectively improved.

Description

A seismic data output method and storage medium for a seismic processing and interpretation system. Technical Field

[0001] This invention belongs to the field of seismic exploration and relates to seismic data output. More specifically, it relates to a seismic data output method and storage medium for a seismic processing and interpretation system. Background Technology

[0002] In the workflow of seismic data processing and interpretation systems, conventional algorithm modules often suffer from long output time when dealing with large volumes of seismic data. Therefore, efficiently outputting seismic data and improving the work efficiency of processing and interpretation personnel is an important issue.

[0003] As shown in Figure 2, existing seismic data processing and interpretation systems use local file systems. Conventional modules operate on a single node. When the data volume is large, the files are divided into multiple small data files according to shot sets. By changing the workflow input and output data, multiple jobs are performed to output multiple small files. Then, the files are manually combined into one large file based on file naming. This not only consumes manpower and has low data output efficiency, but also makes it easy for human errors to occur due to multiple changes to the workflow data. Summary of the Invention

[0004] The purpose of this invention is to solve the problems existing in the prior art and, in view of the shortcomings of the prior art, to provide a method for outputting seismic data in an earthquake processing and interpretation system. This method achieves efficient output of seismic data through the Spark parallel framework and a distributed file system, thereby improving data output efficiency.

[0005] This invention is achieved through the following technical solution:

[0006] In a first aspect, the present invention provides a method for outputting seismic data in a seismic processing and interpretation system, wherein the method comprises the following steps:

[0007] Step 1: Establish a processing flow;

[0008] Step 2: Obtain the parameters input by the user;

[0009] Step 3: Filter the data of the Resilient Distributed Dataset (RDD) based on the parameters input by the user;

[0010] Step 4: Output temporary files for track headers and track data in parallel;

[0011] Step 5: Extract the header information of the header file from the Resilient Distributed Dataset (RDD) and write it to a separate info file;

[0012] Step 6: Merge the temporary track header files to obtain an hdr file as the output file; merge the temporary track data files to obtain a data file as the output file; output the data file, hdr file, and info file.

[0013] Furthermore, the processing flow in step 1 includes two branches:

[0014] Obtain the source data and process it to use as input data;

[0015] Alternatively, obtain the source data and use it as input data.

[0016] Furthermore, the parameters input by the user include: the storage location of the output data, the range of the output data, the header words to be output, and whether to overwrite an existing file with the same name.

[0017] Furthermore, the output data range is the three-dimensional start and end positions required for seismic data output.

[0018] Furthermore, step 3 includes filtering out unnecessary header and file header data and outputting a smaller range based on user input parameters.

[0019] Furthermore, step 4 includes:

[0020] S41. Divide the data to be output into blocks. Each block contains a corresponding header file and a data file. The entire Elastic Distributed Dataset (RDD) contains several header files and several data files, and is output in parallel.

[0021] S42. Perform concat connection on the header file and data file to logically connect all blocks and form a unified whole file to the outside world;

[0022] S43. The entire file is written in parallel to a temporary file on the HDFS distributed file system.

[0023] Furthermore, the temporary file includes a track header temporary file and a track data temporary file; wherein the track header temporary file includes all track headers in the corresponding block, and the track data temporary file includes all track data in the corresponding block.

[0024] Furthermore, step 6 also includes the following steps:

[0025] S61. The temporary files for track headers are named according to the rule name.hdr.001, name.hdr.002...name.hdr.00n; the track data files are named according to the rule name.data.001, name.data.002...name.data.00n.

[0026] S62. Obtain the list of header file names, and concat the header files named name.hdr.002 to name.hdr.00n together with name.hdr.001 in ascending order of file name number;

[0027] S63. Rename the merged name.hdr.001 file to name.hdr;

[0028] S64. Obtain the list of track data file names, and concat the track data files named name.data.002 to name.data.00n together with name.data.001 in ascending order;

[0029] S65. Rename the merged name.data.001 file to name.data.

[0030] Furthermore, Spark's FileOutputCommitter class is used to manage the output files.

[0031] A second aspect of the present invention provides a computer-readable storage medium, characterized in that: the computer-readable storage medium stores at least one computer-executable program, which, when executed by the computer, causes the computer to perform the steps of the seismic data output method based on a seismic processing and interpretation system as described in any one of claims 1-9.

[0032] Compared with the prior art, the beneficial effects of the present invention are: the present invention directly receives the seismic data transferred from the processing and interpretation module in the workflow of the processing and interpretation system, outputs multiple temporary files in blocks and in parallel, and after completion, directly splices the temporary files into a whole data file, which effectively improves the output efficiency of seismic data. Attached Figure Description

[0033] Figure 1 Flowchart of the method of the present invention

[0034] Figure 2. Flowchart of the existing conventional method. Detailed Implementation

[0035] This invention is a method for efficiently outputting seismic data into an internal format using the Spark parallel framework. First, the Spark parallel framework is used to output several trace header and temporary trace data files in a distributed parallel manner. Finally, all trace header and temporary trace data files are concat-merged and joined together, logically achieving efficient output of seismic data on the HDFS distributed file system.

[0036] The basic principle of this method is based on the Spark parallel framework and distributed file system. The invention will be further described in detail below with reference to Figure 1, and it mainly consists of the following steps:

[0037] Example 1:

[0038] Step 1: Establish workflow

[0039] Establish a processing flow, which includes:

[0040] Obtain the source data and process it to use as input data;

[0041] Alternatively, obtain the source data and use it as input data.

[0042] Furthermore, the seismic data format output by this invention can meet the format requirements of different seismic processing software, such as Omega's Dio format.

[0043] Example 2:

[0044] Step 2: Obtain the parameters input by the user.

[0045] In the interactive interface of the output system of this invention, the parameters input by the user are obtained, including the storage location of the output data, the range of the output data, the header words to be output, and whether to overwrite an existing file with the same name.

[0046] Furthermore, the output data range is the three-dimensional start and end positions required for seismic data output, so as to ultimately obtain the required data output.

[0047] Example 3:

[0048] Step 3: Data Filtering

[0049] The filtering operation of the Resilient Distributed Dataset (RDD) is used to filter the data based on the parameters entered by the user in the interface. Specifically, unwanted trace headers and file headers are filtered out, and the output range is narrowed down based on the user input parameters.

[0050] Specifically, the steps for data filtering in a Resilient Distributed Dataset (RDD) include:

[0051] S31. Define the filtering function:

[0052] The filterRecord function filters out trace sets that meet specific conditions. It checks whether the header information (TraceHeaderKey) of each trace matches the conditions defined in the FileFilter object.

[0053] The filterOtherRecord function filters out records that do not meet specific conditions, and its logic is the opposite of filterRecord.

[0054] S32. Record the filtering results:

[0055] The filterRecord function records the records that meet the filtering conditions in Record.

[0056] The filterOtherRecord function records the records that do not meet the filtering criteria in Record.

[0057] The filtered records are still returned in the form of (TraceHeaderKey, Record).

[0058] S33, a filter set with zero filter channels:

[0059] The filter function in Scala is used to further process the filtering results, removing those channel sets with zero channel count, ensuring that the results do not contain empty or invalid datasets.

[0060] Example 4:

[0061] Step 4: Parallel output of track head and track data temporary files

[0062] Leveraging Spark's distributed computing capabilities, the data is processed in chunks. A raw seismic dataset includes a header file (info), multiple trace header files (hdr files), and multiple trace data files (data files), with a one-to-one correspondence between the trace headers and trace data. Each chunk contains multiple trace data, and each trace data includes a trace header and trace data. Specifically, each trace data is retrieved from each chunk's Resilient Distributed Dataset (RDD) chunk in a loop.

[0063] When the Output module receives the data from the Resilient Distributed Dataset (RDD), it includes the block information, meaning that by default, it is divided into blocks of 512MB.

[0064] The specific steps include:

[0065] S41. The data to be output is divided into blocks, and temporary files are output in parallel after block division. Each block outputs a corresponding HDR header file and a data file. The entire Resilient Distributed Dataset (RDD) will have several HDR header files and several data files. Parallel generation and output of all files to the HDFS distributed file system greatly improve efficiency.

[0066] S42. Logically connect the HDR header file and the data file of the block data using concat. This logically connects all blocks to form a unified whole file.

[0067] S43. These processed block data are written in parallel to temporary files on the HDFS distributed file system. The temporary files include an hdr (header) temporary file and a data temporary file. The hdr temporary file contains all the headers of the corresponding block, and the data temporary file contains all the data of the corresponding block.

[0068] Furthermore, Hadoop's OutputCommitter is used to manage the file commit process. The OutputCommitter is responsible for moving temporary files to the final output directory and ensuring data consistency. During parallel writes, data consistency and atomicity of operations must be guaranteed, which is achieved through file locks and atomic operations provided by Hadoop's file system API.

[0069] Example 5:

[0070] Step 5: Extract and output the header file.

[0071] Extract the header information of the hdr header file from the Resilient Distributed Dataset (RDD) and write it to a separate file (info file).

[0072] Specifically, the flatMap method is used to process each record in the RDD, extract the file header information, and merge multiple records into one.

[0073] Furthermore, the filter method is used to filter out records containing file header information.

[0074] Furthermore, the saveAsTextFile method is used to write the collected file header information to a separate file.

[0075] The info file is stored on the HDFS distributed file system, and the path to the file is specified using the `--files` parameter of `spark-submit`.

[0076] Example 6:

[0077] Step 6: Merge temporary files for track headers and track data.

[0078] The Spark FileOutputCommitter class is used to manage output files. Specifically, all temporary header files (hdr header files) corresponding to the output files on the HDFS distributed file system are merged to generate a total header file (hdr file); all temporary data files are merged to generate the final data file. The info file, hdr file, and data file together constitute the final output file package. The user is ultimately provided with a single filename, and this process does not require any physical disk operations for merging.

[0079] Specifically, the merger includes the following steps:

[0080] S61, the hdr header file is named according to the rule name.hdr.001, name.hdr.002...name.hdr.00n; the data file is named according to the rule name.data.001, name.data.002...name.data.00n.

[0081] S62. Obtain the list of HDR header filenames and sort them in ascending order; concat the HDR header files named name.hdr.002, name.hdr.003, etc., to name.hdr.001. The concat method is called through the API provided by the HDFS distributed file system, eliminating the need to move data or read all files and output them as a new file, thus improving efficiency.

[0082] S63. Rename the merged name.hdr.001 file to name.hdr.

[0083] S64. Similarly, obtain the list of data file names in the data channel and sort them in ascending order; concat the data files named name.data.002, name.data.003, etc., into name.data.001.

[0084] S65. Rename the merged name.data.001 file to name.data.

[0085] The above technical solution is only one embodiment of the present invention. For those skilled in the art, based on the principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the technical solutions described in the specific embodiments of the present invention. Therefore, the foregoing description is only a preferred option and is not restrictive.

Claims

1. A method for outputting seismic data in a seismic processing and interpretation system, characterized in that: The method is implemented as follows: Step 1, establish a processing flow; Step 2, obtain the parameters input by the user; Step 3, filter the data of the Resilient Distributed Dataset (RDD) according to the parameters input by the user; Step 4, output the temporary header file and the temporary data file in parallel; Step 5, extract the header information of the temporary header file from the RDD and write it into a separate info file; Step 6, merge the temporary header files to obtain an hdr file as the output file; merge the temporary data files to obtain a data file as the output file; output the data file, hdr file, and info file.

2. The seismic data output method of the seismic processing and interpretation system according to claim 1, characterized in that: The processing flow in step 1 includes: acquiring source data and processing the source data to use as input data; or, acquiring source data and using the source data as input data.

3. The seismic data output method of the seismic processing and interpretation system according to claim 1, characterized in that: The parameters input by the user include: the storage location of the output data, the range of the output data, the header words to be output, and whether to overwrite an existing file with the same name.

4. The seismic data output method of the seismic processing and interpretation system according to claim 3, characterized in that: The output data range is the three-dimensional start and end positions required for seismic data output.

5. The seismic data output method of the seismic processing and interpretation system according to claim 1, characterized in that: The operation in step 3 includes: filtering out the header and file header data that the user does not need, and outputting a small range according to the user input parameters.

6. The seismic data output method of the seismic processing and interpretation system according to claim 1, characterized in that: Step 4 includes: S41, dividing the data to be output into blocks, each block containing a corresponding header file and a data file; the entire Elastic Distributed Dataset (RDD) contains several header files and several data files, which are output in parallel; S42, concatting the header files and data files to logically connect all blocks and form a unified whole file; S43, writing the whole file in parallel to a temporary file on the HDFS distributed file system.

7. The seismic data output method of the seismic processing and interpretation system according to claim 6, characterized in that: The temporary files include a track header temporary file and a track data temporary file; the track header temporary file includes all track headers in the corresponding block, and the track data temporary file includes all track data in the corresponding block.

8. The seismic data output method of the seismic processing and interpretation system according to claim 1, characterized in that: Step 6 further includes the following steps: S61, the temporary track header files are named according to the rule name.hdr.001, name.hdr.002…name.hdr.00n; the track data files are named according to the rule name.data.001, name.data.002…name.data.00n. S62, obtain the list of track header filenames, and concat the track header files named name.hdr.002 to name.hdr.00n in ascending order of filename number to merge them onto name.hdr.001; S63, rename the merged name.hdr.001 file to name.hdr; S64, obtain the list of track data filenames, and concat the track data files named name.data.002 to name.data.00n in ascending order of filename number to merge them onto name.data.001; S65, rename the merged name.data.001 file to name.data.

9. The seismic data output method of the seismic processing and interpretation system according to claim 1 or 8, characterized in that: Use Spark's FileOutputCommitter class to manage the output files.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer-executable program, which, when executed by the computer, causes the computer to perform the steps of the seismic data output method based on the seismic processing and interpretation system as described in any one of claims 1-9.

Images