Seismic data regularization method based on Spark
Through the Spark-based seismic data regularization method, using Spark operators to process and merge data, the problems of large amount of calculation and low efficiency in high-density seismic data processing are solved, and efficient massive data processing and concise parallel computing are achieved, which significantly improves the Productivity.
Patent Information
- Application Number
- CN202110606495.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-31
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2041-05-31
AI Technical Summary
When processing high-density seismic data, the calculation amount and calculation cycle have increased significantly, resulting in low production efficiency, especially when processing massive data, the parallel efficiency is not ideal, and the data regularization method is complex, making it difficult to achieve efficient parallel processing.
Using Spark-based seismic data regularization method, the input data is segmented by setting parameters, and Spark's operators (such as flatMap, mapValue, reduceByKey) are used to process and merge data to achieve regularization of data and slide through the space window. Optimize processing boundaries and reduce network communication and disk IO overhead.
It realizes efficient massive data processing, simplifies parallel computing, improves seismic data processing efficiency, shortens production cycles, significantly improves parallel efficiency, and supports data processing at TB or above.
Smart Images
Figure CN115480298B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of oilfield development, and in particular to a Spark-based seismic data regularization method. Background Art
[0002] In the process of seismic exploration data processing, the degree of data regularization has an important influence on the final imaging accuracy. The more regular the data is, the better the imaging effect is. However, in the actual seismic acquisition process, due to the limitations of factors such as field surface conditions, the layout of the actual field observation system cannot be as regular as the observation system designed indoors. Therefore, it is particularly important to perform indoor regularization processing on the collected irregular data.
[0003] Traditional methods of regularizing seismic data include bin averaging and weighted compensation. With the advancement of high-density seismic technology, the spatial density of collected seismic data is higher, the azimuth is more complete, and the offset is richer. In order to adapt to the characteristics of new data, the five-dimensional regularization method has gradually become a research hotspot. The advancement of regularization methods and the continuous increase in the volume of seismic data have created a new dilemma - the amount of calculation and calculation cycle have increased significantly, affecting the actual production efficiency. Therefore, it is necessary to adopt a parallel method based on a multi-computing node cluster to improve efficiency. MPI is widely used in the field of seismic exploration to achieve multi-process parallelism. However, algorithms such as data regularization are more complicated in terms of the organization of data input and the aggregation of data output. It is very difficult to use MPI for parallelization, and because a large amount of data network transmission overhead and temporary file read and write operations need to be introduced, the improvement of parallel efficiency is not ideal.
[0004] Spark is a new type of high-speed, general-purpose big data computing and processing engine with many advantages such as high scalability, high storage efficiency, and in-memory computing. Spark is based on the idea of data distribution and parallelism, which partitions data reasonably and processes each partition in parallel. Spark provides a variety of powerful operation operators, including one-to-one, many-to-one, and many-to-many input and output partitions. These operators realize parallel processing of data, and the processing details are transparent to developers. Based on the Spark engine, developers do not have to pay too much attention to the parallel details between computing nodes such as data network transmission, memory and disk usage, and the development complexity is lower.
[0005] Existing technical solutions are all based on MPI for parallel processing. These solutions are very complicated when processing complex data input and complex data aggregation, and introduce a large number of temporary file read and write operations and data network transmission operations, resulting in unsatisfactory efficiency improvement. These problems are more significant when dealing with massive data.
[0006] In the Chinese patent application with application number: 201811382738.3, an entropy-constrained data-driven regular framework seismic data regularization method is involved, which includes the following steps: graying the original irregular seismic data; dividing the grayed seismic data into data subsets, and calculating the entropy of each data subset; calculating the key value of each data subset according to the entropy of each data subset; selecting data subsets with larger key values to form a training set, performing dictionary training, and obtaining an adaptive dictionary basis function; using the relative error of the iterative reconstruction result and the maximum number of iterations as the convergence criteria, and using the finally trained adaptive dictionary basis function to interpolate and encrypt the irregular seismic data; outputting the interpolated seismic data on a regular grid to complete the regularization of the seismic data.
[0007] In the Chinese patent application with application number: 201310439558.5, a high-dimensional seismic data regularization method is involved, which belongs to the field of seismic data preprocessing in oil and gas exploration and development. The method includes: (1) inputting an original three-dimensional shot gather; (2) transforming the original three-dimensional shot gather in the time-space domain into the frequency-space domain by using a high-dimensional Fourier transform, and then sorting to obtain a five-dimensional shot gather; (3) optimizing a frequency slice to obtain a non-aliased frequency spectrum; (4) judging whether the iteration threshold value is reached, if yes, proceeding to step (5), if not, returning to step (3); (5) transforming the non-aliased frequency spectrum into the time-space domain by using a high-dimensional inverse Fourier transform, that is, obtaining regular shot gather data; (6) outputting the regular shot gather data.
[0008] In the Chinese patent application with application number: 201110182710.7, a method for regularizing high-dimensional seismic data under Radon spectrum constraints is involved, including: collecting raw data of a plane wave front: calculating the covariance matrix CD of the data space corresponding to the seismic data; calculating the covariance matrix CM of the model space between the energies of each wave number component in the seismic data (f, k) spectrum; constructing an objective function using the covariance matrix CD of the data space and the covariance matrix CM of the model space obtained above, and calculating the regularized seismic data.
[0009] The above existing technologies innovate the specific algorithms or processes for seismic data regularization, but do not involve parallel implementation methods of data regularization methods, as well as implementation strategies for dealing with massive data processing, which are quite different from the present invention and fail to solve the technical problems we want to solve. Therefore, we have invented a new Spark-based seismic data regularization method. Summary of the invention
[0010] The purpose of the present invention is to provide a Spark-based seismic data regularization method with the advantages of higher efficiency, simplicity in implementation, ease of maintenance, and support for massive data processing.
[0011] The object of the present invention can be achieved by the following technical measures: a Spark-based seismic data regularization method, the Spark-based seismic data regularization method comprising:
[0012] Step 1, setting parameters for regularization processing;
[0013] Step 2: Slice the input gather data according to the parameters, and obtain several data slices to form the input data RDD (RDD is the abbreviation of Resilient Distributed Dataset);
[0014] Step 3, process the input data RDD to obtain the regularized data RDD;
[0015] Step 4: average and merge the regularized data RDD to obtain the result data RDD.
[0016] The purpose of the present invention can also be achieved by the following technical measures:
[0017] In step 1, the parameters set for regularization processing include the input data range, the output data range to be regularized, the spatial window range, and the spatial window sliding mode.
[0018] In step 1, the spatial window sliding method adopts two methods: non-overlapping and overlapping.
[0019] In step 2, the gather data within the output data range are divided according to the spatial window parameters.
[0020] In step 2, in order to improve the processing effect of the boundary, the output data range is expanded outward by no more than one spatial window as the range of the spatial window sliding; if the spatial window sliding range exceeds the range of the input data, the data at the boundary of the input data is selected to replace the required data that exceeds the range.
[0021] In step 2, if the non-overlapping spatial window sliding method is adopted, the range of the spatial window sliding is divided into multiple data slices according to the size of the spatial window parameters, and each slice contains several track data; non-overlapping spatial window sliding is equivalent to sliding a spatial window each time to form a data slice, and there is no repeated input data between adjacent data slices.
[0022] In step 3, the flatMap operator is applied, and the following processing is performed in the flatMap operator:
[0023] (3.1) Apply the data regularization method based on random sampling to process the data in each data slice of RDD;
[0024] (3.2) Convert the data obtained by the data regularization method based on random sampling into the spatial domain and the temporal domain to obtain the regularized gather data corresponding to the input data RDD;
[0025] (3.3) Mark each gather data in the partition with a sorting keyword;
[0026] The flatMap operator will flatten the processing results, that is, expand the result data slices, and the regularized gather data obtained after expansion will form a new RDD, which is recorded as the regularized data RDD.
[0027] Step 3.2 also includes calculating and setting the trace header for the regularized trace data.
[0028] In step 3.3, if interpolation processing is to be performed, the original data labels of the input data are recorded, and then the corresponding labeled data in the regularized data are replaced with the original data.
[0029] Step 4 includes:
[0030] (4.1) Apply the mapValue operator to add a count mark of 1 to each gather in the regularized data RDD;
[0031] (4.2) Apply the reduceByKey operator to sum and merge the trace data and count marks with the same mark, that is, add the corresponding trace data samples and count marks;
[0032] (4.3) Apply the mapValue operator to divide all sample points of the merged data set by the corresponding counting mark and remove the counting mark.
[0033] Step 4.3 also includes sorting the processed data.
[0034] The Spark-based seismic data regularization method in the present invention is simpler to implement and easier to maintain than the prior art; the network communication and disk IO overheads are smaller during parallel computing, and the parallel efficiency is higher; it can support efficient processing of massive data above TB level, which can greatly improve the processing efficiency of seismic data and shorten the production cycle. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a flowchart of a specific embodiment of the Spark-based seismic data regularization method of the present invention. DETAILED DESCRIPTION
[0036] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.
[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations and / or combinations thereof.
[0038] The Spark-based seismic data regularization method of the present invention includes the following steps:
[0039] (1) Setting parameters for regularization processing, including input data range, output data range to be regularized, spatial window range, spatial window sliding mode, etc. The spatial window sliding mode adopts overlapping mode.
[0040] (2) The input data set is divided according to the parameters, and the obtained data slices constitute the input data RDD. The specific division strategy is: divide the data set within the output data range according to the spatial window parameters. In order to improve the processing effect of the boundary, the output data range is generally extended outward by no more than one spatial window as the range of the spatial window sliding. If the spatial window sliding range exceeds the range of the input data, the data at the boundary of the input data is selected to replace the required data that exceeds the range. The overlapping spatial window sliding method is adopted, and half of the spatial window is slid each time to form a data slice.
[0041] (3) Process the input data RDD to obtain the regularized data RDD. The specific steps are: Apply the flatMap operator and perform the following processing in the flatMap operator:
[0042] (3.1) Apply a random sampling based data regularization method to the data in each data slice of the RDD.
[0043] (3.2) The data obtained by the data regularization method based on random sampling is converted into the spatial domain and the temporal domain to obtain the regularized gather data corresponding to the input data RDD.
[0044] Furthermore, the trace header is calculated and set for the regularized trace data.
[0045] (3.3) Use the sorting key to mark each gather data in the partition. For example, if the input data is a CMP gather, use the horizontal and vertical line numbers to mark it.
[0046] The flatMap operator will flatten the processing results, that is, expand the result data slices, and the regularized gather data obtained after expansion will form a new RDD, which is recorded as the regularized data RDD.
[0047] (4) Average and merge the regularized data RDD to obtain the result data RDD. The specific steps are:
[0048] (4.1) Apply the mapValue operator to add a count mark of 1 to each gather in the regularized data RDD;
[0049] (4.2) Apply the reduceByKey operator to sum and merge the trace data and count marks with the same tag, that is, add the corresponding trace data samples and count marks.
[0050] (4.3) Apply the mapValue operator to divide all sample points of the merged data set by the corresponding counting mark and remove the counting mark.
[0051] Furthermore, the processed data can be sorted.
[0052] Embodiment 1:
[0053] In a specific embodiment 1 of the present invention, Figure 1 As shown, Figure 1 The flowchart of the Spark-based seismic data regularization method of the present invention is shown in FIG. The Spark-based seismic data regularization method includes the following steps:
[0054] (1) Setting parameters for regularization processing, including input data range, output data range to be regularized, spatial window range, spatial window sliding mode, etc. The spatial window sliding mode adopts overlapping mode.
[0055] (2) The input gather data is segmented according to the parameters, and the obtained data slices constitute the input data RDD. In the figure, only three data slices (or partitions) are drawn as examples.
[0056] (3) Apply the flatMap operator to process the input data RDD and obtain the regularized data RDD.
[0057] (4) Apply the mapValue operator to add a count mark of 1 to each gather in the regularized data RDD;
[0058] (5) Apply the reduceByKey operator to sum and merge the trace data and count marks with the same mark, that is, add the corresponding trace data samples and count marks.
[0059] (6) Apply the mapValue operator to divide all sample points of the merged gather data by the corresponding count mark and remove the count mark.
[0060] Figure 1 The large rectangular box in the middle represents the data RDD. Each small rectangular box in the large rectangular box represents a partition of the RDD. The figure only uses three partitions as an example. The middle of the two large rectangular boxes represents the conversion of the RDD. The English words below indicate the operators used in the conversion, and the Chinese words above indicate the corresponding algorithm process. The arrows indicate the flow of data during the conversion process.
[0061] Embodiment 2:
[0062] In the specific embodiment 2 of the present invention, data regularization processing based on non-overlapping sliding mode is implemented, which includes the following steps:
[0063] (1) Setting parameters for regularization processing, including input data range, output data range to be regularized, spatial window range, spatial window sliding mode, etc. The spatial window sliding mode adopts a non-overlapping mode.
[0064] (2) The input data set is divided according to the parameters, and the obtained data slices constitute the input data RDD. The specific division strategy is: divide the data set within the output data range according to the spatial window parameters. In order to improve the processing effect of the boundary, the output data range is generally extended outward by no more than one spatial window as the range of the spatial window sliding. If the spatial window sliding range exceeds the range of the input data, the data at the boundary of the input data is selected to replace the required data that exceeds the range. The non-overlapping spatial window sliding method is adopted, and the spatial window sliding range is divided into multiple data slices according to the size of the spatial window parameters, and each slice contains several data sets. Non-overlapping spatial window sliding is equivalent to sliding a spatial window each time to form a data slice, and there is no repeated input data between adjacent data slices.
[0065] (3) Process the input data RDD to obtain the regularized data RDD. The specific steps are: Apply the flatMap operator and perform the following processing in the flatMap operator:
[0066] (3.1) Apply a random sampling based data regularization method to the data in each data slice of the RDD.
[0067] (3.2) The data obtained by the data regularization method based on random sampling is converted into the spatial domain and the temporal domain to obtain the regularized gather data corresponding to the input data RDD.
[0068] Furthermore, the trace header is calculated and set for the regularized trace data.
[0069] (3.3) Use the sorting key to mark each gather data in the partition. For example, if the input data is a CMP gather, use the horizontal and vertical line numbers to mark it.
[0070] The flatMap operator will flatten the processing results, that is, expand the result data slices, and the regularized gather data obtained after expansion will form a new RDD, which is recorded as the regularized data RDD.
[0071] (4) Since a non-overlapping spatial window sliding method is used, there is no duplicate data in each partition of the regularized data RDD, and the regularized data RDD can be output without performing an averaging operation. The data can be sorted before output.
[0072] Embodiment 3:
[0073] In the specific embodiment 3 of the present invention, data interpolation processing based on overlapping sliding mode is implemented, which includes the following steps:
[0074] (1) Setting parameters for regularization processing, including input data range, output data range to be regularized, spatial window range, spatial window sliding mode, etc. The spatial window sliding mode adopts overlapping mode.
[0075] (2) The input data set is divided according to the parameters, and the obtained data slices constitute the input data RDD. The specific division strategy is: divide the data set within the output data range according to the spatial window parameters. In order to improve the processing effect of the boundary, the output data range is generally extended outward by no more than one spatial window as the range of the spatial window sliding. If the spatial window sliding range exceeds the range of the input data, the data at the boundary of the input data is selected to replace the required data that exceeds the range. The overlapping spatial window sliding method is adopted, and half of the spatial window is slid each time to form a data slice. There will be repeated input data between adjacent data slices.
[0076] (3) Process the input data RDD to obtain the regularized data RDD. The specific steps are: Apply the flatMap operator and perform the following processing in the flatMap operator:
[0077] (3.1) Record the original data labels of the input data and obtain the label set S.
[0078] (3.2) Apply a random sampling based data regularization method to the data in each data slice of the RDD.
[0079] (3.2) The data obtained by the data regularization method based on random sampling is converted into the spatial domain and the temporal domain to obtain the regularized gather data corresponding to the input data RDD.
[0080] Furthermore, the trace header is calculated and set for the regularized trace data.
[0081] (3.3) Use the sorting key to mark each gather data in the partition. For example, if the input data is a CMP gather, use the horizontal and vertical line numbers to mark it.
[0082] (3.4) According to the label set S, the original data is used to replace the corresponding labeled data in the regularized data.
[0083] The flatMap operator will flatten the processing results, that is, expand the result data slices, and the regularized gather data obtained after expansion will form a new RDD, which is recorded as the interpolation data RDD.
[0084] (4) Average and merge the interpolation data RDD to obtain the result data RDD. The specific steps are:
[0085] (4.1) Apply the mapValue operator to add a count mark of 1 to each gather in the interpolation data RDD;
[0086] (4.2) Apply the reduceByKey operator to sum and merge the trace data and count marks with the same tag, that is, add the corresponding trace data samples and count marks.
[0087] (4.3) Apply the mapValue operator to divide all sample points of the merged data set by the corresponding counting mark and remove the counting mark.
[0088] Furthermore, the processed data can be sorted.
[0089] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions recorded in the aforementioned embodiments or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
[0090] Except for the technical features described in the specification, all other technical features are known technologies to those skilled in the art.
Claims
1. A Spark-based seismic data regularization method, characterized in that: The Spark-based seismic data regularization method includes: Step 1, setting parameters for regularization processing; Step 2, split the input gather data according to the parameters, and the obtained data slices constitute the input data RDD; Step 3, process the input data RDD to obtain the regularized data RDD; Step 4, average and merge the regularized data RDD to obtain the result data RDD; In step 1, the parameters set for regularization processing include the input data range, the output data range to be regularized, the spatial window range, and the spatial window sliding mode; In step 1, the spatial window sliding mode adopts two modes: non-overlapping or overlapping; In step 2, the gather data within the output data range is divided according to the spatial window parameters; In step 2, the output data range is extended outward by no more than one spatial window as the range of the spatial window sliding; if the spatial window sliding range exceeds the range of the input data, the data at the boundary of the input data is selected to replace the required data that exceeds the range; In step 3, the flatMap operator is applied, and the following processing is performed in the flatMap operator: (3.1) Apply a data regularization method based on random sampling to process the data in each data slice of the RDD; (3.2) Convert the data obtained by the data regularization method based on random sampling into the spatial domain and the temporal domain to obtain the regularized gather data corresponding to the input data RDD; (3.3) Mark each gather data in the partition with a sorting key; The flatMap operator will flatten the processing results, that is, expand the result data slices, and the regularized gather data obtained after expansion will form a new RDD, which is recorded as the regularized data RDD.
2. The Spark-based seismic data regularization method according to claim 1, characterized in that: In step 2, if the non-overlapping spatial window sliding method is adopted, the range of the spatial window sliding is divided into multiple data slices according to the size of the spatial window parameters. Each slice contains several track data, which is equivalent to sliding a spatial window each time to form a data slice, and there is no repeated input data between adjacent data slices; if the overlapping spatial window sliding method is adopted, half of the spatial window is slid each time to form a data slice, and there will be repeated input data between adjacent data slices.
3. The Spark-based seismic data regularization method according to claim 2, characterized in that: Step 3.2 also includes calculating and setting the trace header for the regularized trace data.
4. The Spark-based seismic data regularization method according to claim 2, characterized in that: In step 3.3, interpolation processing is performed to record the original data labels of the input data, and then the original data is used to replace the corresponding labeled data in the regularized data.
5. The Spark-based seismic data regularization method according to claim 1, characterized in that: Step 4 includes: (4.1) Apply the mapValue operator to add a count marker of 1 to each track set in the regularized data RDD; (4.2) Apply the reduceByKey operator to sum and merge the trace data and count marks with the same mark, that is, add the corresponding trace data samples and count marks; (4.3) Apply the mapValue operator to divide all sample points of the merged data set by the corresponding count mark and remove the count mark.
6. The Spark-based seismic data regularization method according to claim 5, characterized in that: Step 4.3 also includes sorting the processed data.