Parallel granularity division method for seismic data based on spark and application thereof

Abstract

The application belongs to the field of seismic exploration data processing, and particularly relates to a parallel granularity division method of seismic data based on Spark and application thereof. The number of non-overlapping windows is determined on trace data RDD, and the non-overlapping windows are divided. Whether each trace data in the non-overlapping window should be an overlapping trace is judged. Other non-overlapping windows to which the overlapping trace belongs are calculated. Each non-overlapping window is expanded into a complete window according to the overlapping trace, so as to convert the trace data RDD into a window body RDD. The parallel granularity of the window body RDD is the trace data in a complete window. The application can accurately and clearly divide a three-dimensional data body into multiple small overlapping three-dimensional bodies, and can ensure that the parallel granularity is optimized and smaller when parallel calculation is performed by using a spark framework.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of seismic exploration data processing, and particularly relates to a parallel granularity division method for seismic data based on Spark and application thereof. BACKGROUND

[0002] RDD (Resilient Distributed Dataset) is a basic data structure of Spark, representing an immutable, partitionable, and parallel-computable collection. In seismic data processing, if seismic data is processed channel by channel, the seismic data can be directly converted into channel data RDD under the spark framework; if the entire channel set is processed, it can be converted into channel set RDD. The two cases can be completed through the relatively simple RDD flow of the flatMap function in spark. When processing stacked data, sometimes the data needs to be processed through a sliding window, and there is an overlap between the windows, which increases the difficulty of three-dimensional division of the data. SUMMARY

[0003] The present application aims to solve the problems in the prior art and provide a parallel granularity division method for seismic data based on Spark.

[0004] The present application is implemented through the following technical scheme: a parallel granularity division method for seismic data based on Spark, comprising the following steps:

[0005] Determine the number of non-overlapping windows on the channel data RDD, and divide the non-overlapping windows;

[0006] Determine whether each channel data in the non-overlapping window should be an overlapping channel;

[0007] Calculate other non-overlapping windows to which the overlapping channel belongs;

[0008] Expand each non-overlapping window into a complete window according to the overlapping channel, thereby converting the channel data RDD into a window RDD;

[0009] The parallel granularity of the window RDD is the channel data in a complete window.

[0010] Further, the method for constructing the channel data RDD comprises the following steps:

[0011] Input the seismic data into a distributed file system for partitioning, and each partition corresponds to the data of multiple lines, and the data of each line is composed of multiple channel data;

[0012] According to the data in the partition, an Inline number RDD is constructed, each element in the Inline number RDD corresponds to a key-value pair expressed as key1-value1, key1 takes the value of InlineNum, which is the line number in the Inline direction, and value1 takes the value of all trace data of the common Inline number;

[0013] The Inline number RDD is converted into a trace data RDD, each element in the trace data RDD corresponds to a key-value pair expressed as key2-value2, key2 takes the value of (InlineNum, CrosslineNum), and value2 takes the value of the trace data corresponding to key2, InlineNum represents the line number in the Inline direction, and CrosslineNum represents the line number in the Crossline direction.

[0014] Further, by using a flatMap conversion function, the Inline number and the Crossline number are extracted from the trace header as the value of key2 for each trace data in each key1-value1 in the Inline number RDD.

[0015] Further, the number of non-overlapping windows is determined as follows:

[0016] The number of Inline direction blocks nInline is calculated according to the following formula:

[0017] nInline = (InlineN - OverlapInline) / (WinInline - OverlapInline) + 1

[0018] In the formula, InlineN represents the total number of lines in the Inline direction, WinInline represents the number of traces in the Inline direction that a complete window should have, and OverlapInline represents the number of overlapping traces in the Inline direction within a single overlapping window.

[0019] The number of Crossline direction blocks nCrossline is calculated according to the following formula:

[0020] nCrossline = (CrossN - OverlapCrossline) / (WinCrossline - verlapCrossline) + 1

[0021] In the formula, CrossN represents the total number of Crossline directions, WinCrossline represents the number of traces in a complete window in the Crossline direction, and OverlapInline represents the number of overlapping traces in the Crossline direction in a single overlapping window.

[0022] The product of the number of blocks nInline and the number of blocks nCrossline is the number of windows of the non-overlapping window.

[0023] Further, the window position of the non-overlapping window is determined by the window position iInline in the Inline direction and the window position icrossline in the Crossline direction, and iInline and icrossline are numbered from 0.

[0024] Further, the key-value pair corresponding to each element in the window RDD is represented as key3-value3, key3 takes the value of the window number, and value3 takes the value of all trace data in the complete window corresponding to key3.

[0025] key3 is calculated according to the following formula:

[0026] key3 = iInline * nCrossline + icrossline

[0027] In the formula, iInline represents the window position in the Inline direction, icrossline represents the window position in the Crossline direction, and nCrossline represents the number of blocks in the Crossline direction.

[0028] Further, before determining whether it is an overlapping trace, it is marked first:

[0029] For each trace data, first mark flagInline = 0, flagCrossline = 0;

[0030] Then calculate its window position (iInline, icrossline);

[0031] When iInline >= 1, icrossline >= 1, determine whether the trace is an overlapping trace:

[0032] If InlineNum <= (iInline-1) * (WinInline-OverlapInline) + WinInline-1, mark flagInline =-1 in the Inline direction;

[0033] If CrosslineNum <= (iCrossline-1)*(WinCrossline-OverlapCrossline)++WinCrossline-1, mark flagCrossline=-1 in Crossline direction.

[0034] Further, according to the mark, judge whether it is an overlapping trace and belongs to a non-overlapping window:

[0035] If flagInline+flagCrossline==0, flagInline=0, flagCrossline=0, which means that the trace data is not an overlapping trace.

[0036] Otherwise, if flagInline*flagCrossline==0, flagInline=0, flagCrossline=-1, which means that the trace data belongs to the overlapping area in Crossline direction; flagInline=-1, flagCrossline=0, which means that the trace data belongs to an overlapping area in Inline direction, so the trace data in the overlapping area also belongs to the window: (iInline+flagInline)*nCrssline+(iCrossline+flagCrssline).

[0037] If flagInline*flagCrossline!=0, flagInline=-1, flagCrossline=-1, which means that the trace data belongs to the overlapping area in both Inline and Crossline directions, so the trace data in the overlapping area also belongs to the window: (iInline+flagInline)*nCrssline+(iCrossline+flagCrssline), (iInline+0)*nCrossline+(iCrossline+flagCrssline), (iInline+flagInline)*nCrossline+(iCrossline+0).

[0038] The application also provides an application of a parallel granularity division method of seismic data based on Spark, characterized by being used for dividing pre-stack seismic data, post-stack seismic data, three-dimensional seismic data or two-dimensional seismic data.

[0039] Further, after the data processing according to the parallel granularity is completed, if output is needed according to the Inline number RDD, the window RDD is first converted into a trace data RDD with (InlineNum, CrosslineNum) as the key value, and the data is de-duplicated, and then the Inline is converted into an RDD with Inline as the key value.

[0040] Compared with the prior art, the beneficial effects of the present application include:

[0041] The present application can accurately and clearly divide a three-dimensional data body into a plurality of small overlapping three-dimensional bodies, and ensure that the parallel granularity is optimized when parallel computing is performed using a spark framework, so that the parallel granularity is smaller. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A schematic diagram of the construction of the seismic data input into the distributed system;

[0043] Figure 2 A schematic diagram of the trace data arranged according to the Crossline number for the same Inline number;

[0044] Figure 3 A schematic diagram of the three-dimensional seismic data divided into small three-dimensional bodies;

[0045] Figure 4 A schematic diagram of different windows;

[0046] Figure 5 A diagram of the relationship between the window position and the window number;

[0047] Figure 6 A schematic diagram of the marked overlapping area. DETAILED DESCRIPTION

[0048] A parallel granularity division method for seismic data based on Spark, comprising the following steps:

[0049] Determine the number of non-overlapping windows on the trace data RDD, and divide the non-overlapping windows;

[0050] Determine whether each trace data in the non-overlapping window should be an overlapping trace;

[0051] Calculate the other non-overlapping windows to which the overlapping trace belongs;

[0052] According to the overlapping trace, each non-overlapping window is expanded into a complete window, so as to convert the trace data RDD into a window body RDD;

[0053] The parallel granularity of the window body RDD is the trace data in a complete window.

[0054] This invention can be used in scenarios where seismic data needs to be re-segmented during processing, and can be applied to both pre-stack and post-stack seismic data; it can be applied to both 3D and 2D seismic data.

[0055] This specific implementation method uses three-dimensional post-stack seismic data as an example for illustration.

[0056] The present invention will now be described in further detail with reference to the accompanying drawings.

[0057] This specific implementation demonstrates a method for dividing post-stack seismic data in the Inline and Crossline directions. The time direction is assumed to be a complete data stream without any division. If further division in the time direction is required, the same marking method can be used to mark the overlapping data in the time direction.

[0058] First, refer to Figure 1 As shown, seismic trace data is input into a distributed file system, and the input data is partitioned, with each partition corresponding to data from multiple lines. The format of post-stack seismic data is stored by trace, with each line consisting of multiple traces, and each 3D data volume consisting of multiple lines. Data within the same inline line resides in the same partition.

[0059] refer to Figure 2 As shown, a single data point is determined by (Inline, Crossline). The number of channels in the Inline direction is the number of Inline lines. Data points with the same Inline number are arranged according to their Crossline number, forming the data within an Inline line.

[0060] refer to Figure 3 As shown, the 3D seismic data is divided into small 3D volumes of size WinInline*WinCrossline, with single-sided overlap window lengths of OverlapInline and OverlapCrossline, respectively.

[0061] Example 1

[0062] The method for constructing the aforementioned data RDD includes the following steps:

[0063] The earthquake data is entered into a distributed file system for partitioning. Each partition corresponds to multiple data lines, and each data line consists of multiple data channels.

[0064] Based on the data in the partition, construct an Inline RDD. Each key-value pair in the Inline RDD is represented as key1-value1, where key1 is InlineNum, which is the line number in the Inline direction, and value1 is the data of all tracks in the Inline RDD.

[0065] The Inline number RDD is converted into a trace data RDD, and each element in the trace data RDD is expressed as a key2-value2 pair, where key2 takes the value of (InlineNum, CrosslineNum), and value2 takes the value of the trace data corresponding to key2. InlineNum represents the line number in the Inline direction, and CrosslineNum represents the line number in the Crossline direction.

[0066] (1) An RDD is constructed with the key value being the Inline number and the value being all the trace data corresponding to the line number. The specific method is as follows: the Inline number and the trace data of each trace data are extracted, and all the trace data corresponding to the same Inline number are found. Each element corresponds to a key-value pair.

[0067] (2) The Inline number RDD is converted into a trace data RDD, and the key value of the trace data RDD is a combined key (Inline, Crossline), and the value is a trace data (trace header + trace data). The flatMap conversion function is used to extract the Inline number and the Crossline number from the trace header for each trace data in each key-value pair (i.e., each line) in the Inline number RDD, and the extracted Inline number and Crossline number are used as the key value, and the corresponding trace data is used as the value.

[0068] Embodiment 2

[0069] The number of windows of non-overlapping windows is determined as follows:

[0070] The number of blocks in the Inline direction nInline is calculated according to the following formula:

[0071] nInline = (InlineN - OverlapInline) / (WinInline - OverlapInline) + 1

[0072] In the formula, InlineN represents the total number of lines in the Inline direction, WinInline represents the number of traces in the Inline direction that a complete window should have, and OverlapInline represents the number of overlapping traces in the Inline direction within a single overlapping window.

[0073] The number of blocks in the Crossline direction nCrossline is calculated according to the following formula:

[0074] nCrossline = (CrossN - OverlapCrossline) / (WinCrossline - OverlapCrossline) + 1

[0075] In the formula, CrossN represents the total number of lines in the Crossline direction, WinCrossline represents the number of lines in the Crossline direction that a complete window should have, and OverlapInline represents the number of overlapping lines in the Crossline direction within a single overlapping window.

[0076] The product of the number of blocks nInline and the number of blocks nCrossline is the number of windows of the non-overlapping windows.

[0077] The specific method is as follows:

[0078] Referring to Figure 4 As shown in the figure, the number of blocks nInline in the Inline direction and the number of blocks nCrossline in the Crossline direction are calculated according to the non-overlapping condition between windows. As shown in the figure, Figure 4 nInline = 3 and nCrossline = 5.

[0079] The specific calculation process is as follows: there are 8 lines in the Inline direction, and the number of lines in the Crossline direction is 9 (as shown in the figure). In each block window, there are 5 lines of data in the Inline direction (i.e. WinInline = 5), and there are 3 lines of data in the Crossline direction (WinCrossline = 3). The overlapping condition between windows is that the number of overlapping lines in the Inline direction is 2 (OverlapInline), and the number of overlapping lines in the Crossline direction is 1 (OverlapCrossline). Then nInline = (8-2) / (5-2)+1 = 3, and nCrossline = (9-1) / (3-1)+1 = 5. A total of 3*5 = 15 non-overlapping windows are divided, and the window number starts from 0. Figure 4 Example 3

[0080] Before determining whether it is an overlapping line, marking is performed first:

[0081]

[0082] ​First, calculate the window position in Inline direction iInline(iInline = InlineNum / (WinInline-OverlapInline)), and Crossline direction icrossline(iCrossline = CrosslineNum / (WinCrossline-OverlapCrossline)), (iInline, icrossline start from 0). The trace data belongs to window(iInline*nCrossline+icrossline)(key3 value). For example Figure 5 In the middle window 10, the key value of trace data in the window is 10. After dividing the trace data into windows (here refers to the key value of RDD to which each trace data belongs), the window position in Inline direction iInline = 1, and the window position in Crossline direction icrossline = 2.

[0083] Figure 5 In the right window, the window is the narrowest, which is to show that the last several trace data may not meet a normal window size.

[0084] The final divided window will overlap, and the non-overlapping window is introduced, which is only to calculate the number of windows according to the non-overlapping part and to facilitate the division of data. Figure 4 As shown, the middle is a complete window, and the left is the data after removing the overlapping part.

[0085] Given the number of windows, iInline, icrossline can be calculated, iInline = key / nCrossline, icrossline = key-nCrossline*iInline.

[0086] Then, it is judged whether the trace data is an overlapping trace. If it is an overlapping trace, it is judged which window the overlapping trace belongs to (for example, in the final divided RDD, Figure 6 In the middle overlapping area ①, it belongs to window 32 and 33, Figure 6 In ②, it belongs to window 25, 26, 33 and 34.

[0087] Set a flag: the overlapping area in Inline direction is set as flagInline = -1, and the overlapping area in Crossline direction is set as flagCrosseline = -1. If the trace data is not in the overlapping area, flagInline = 0, and flagCrossline = 0.

[0088] The overlapping area is determined by calculation. For each trace, first mark flagInline=0, flagCrossline=0, then calculate its position (iInline, iCrossline). When iInline>=1, iCrossline>=1, determine whether the trace is an overlapping trace:

[0089] If InlineNum<= (iInline-1)*(WinInline-OverlapInline)+WinInline-1, then flagInline=-1 If CrosslineNum<= (iCrossline-1)*(WinCrossline-OverlapCrossline)+WinCrossline-1, then flagCrossline=-1.

[0090] Example 4

[0091] According to the mark, determine whether it is an overlapping trace and the non-overlapping window it belongs to, convert the trace data RDD to a three-dimensional window RDD.

[0092] Next, determine whether flagInline+flagCrossline==0, if so, flagInline=0, flagCrossline=0, which means the trace data is not an overlapping trace;

[0093] Otherwise, if flagInline*flagCrossline==0, then flagInline=0, flagCrossline=-1, which means the trace data belongs to the Crossline direction overlapping area. flagInline=-1, flagCrossline=0, which means the trace data belongs to an Inline overlapping area, so the overlapping area trace data also belongs to the window: (iInline+flagInline)*nCrssline+(iCrossline+flagCrssline);

[0094] If flagInline*flagCrossline≠0, then flagInline=-1, flagCrossline=-1, which means the trace belongs to the overlapping area of two directions, so the overlapping area also belongs to the window: (iInline+flagInline)*nCrossline+(iCrossline+flagCrossline), (iInline+0)*nCrossline+(iCrossline+flagCrossline), (iInline+flagInline)*nCrossline+(iCrossline+0).

[0095] The flatMap is used to build an RDD with key3 value as window number (window number) and value3 value as trace data in a window. The window is not a time window, and it is not to cut a trace data RDD into multiple three-dimensional window RDDs, but to recombine the trace data RDD into a three-dimensional window RDD. The window is composed of multiple adjacent trace data, such as the data in the red box in the following figure, which is in a window. It can be understood that there are multiple trace data from different Inline and crossline in a window number Figure 2

[0096] Thus, all the trace data in each overlapping window is obtained. An RDD with key3 value as window number and value3 value as all the trace data in a window is built. At this time, the data can be processed in parallel, and the parallel granularity is the data of a complete window with overlapping data.

[0097] Each complete window contains overlapping and non-overlapping parts. Overlapping and non-overlapping are only for window division, and the processing method of the data in the window is the same after the window division is completed.

[0098] Example 5

[0099] After the data processing according to the parallel granularity is completed, if it is required to output according to the Inline number RDD, the window RDD is first converted into a trace data RDD with (InlineNum, CrosslineNum) as key value, and the data is de-duplicated, and then it is converted into an RDD with Inline as key value. Finally, the RDD is directly output or passed to the next module for processing.

[0100] Example 6

[0101] (1) The seismic trace data is input into the distributed file system, and the input data is partitioned, and each partition corresponds to the data of multiple lines.

[0102] ​(2) Construct the RDD whose key value is Inline number and value is all trace data corresponding to the Inline number.

[0103] (3) Convert the Inline number RDD to trace data RDD, whose key value is the combined key (Inline, Crossline) and value is a trace data (trace header + trace data).

[0104] (4) Convert the trace data RDD to window RDD. Use flatMap to construct the RDD whose key value is window number and value is the trace data in a window. First, calculate the total window number to be divided, and then calculate which window each trace data belongs to without overlapping.

[0105] (5) Then, judge whether the data is overlapping trace and set a flag. Finally, divide the overlapping trace into another window according to the flag. At this time, the data can be processed in parallel, and the parallel granularity is one window of data.

[0106] (6) After the data processing is completed, if it is needed to output according to the Inline number RDD, first convert the window RDD to the trace data RDD whose key value is (Inline, Crossline) and remove the duplicate data. Then, convert it to the RDD whose key value is Inline number.

[0107] The technical solutions described above are only specific embodiments of the present application. For those skilled in the art, on the basis of the principles disclosed in the present application, various types of improvements or modifications can be easily made, and are not limited to the technical solutions described in the above embodiments. Therefore, the above description is only preferred, and is not limited in nature.

Claims

1. A parallel grain partitioning method for seismic data based on Spark, characterized in that, The method comprises the following steps: determining the number of non-overlapping windows on the trace data RDD, and dividing the non-overlapping windows; determining whether each trace data in the non-overlapping window is an overlapping trace; calculating other non-overlapping windows to which the overlapping trace belongs; extending each non-overlapping window to a complete window according to the overlapping trace, so as to convert the trace data RDD into a window RDD; the parallel granularity of the window RDD is the trace data in a complete window; the number of non-overlapping windows is determined as follows: the number of blocks nInline in the Inline direction is calculated as follows: nInline = (InlineN- OverlapInline) / (WinInline – OverlapInline)+1 where InlineN represents the total number of lines in the Inline direction, WinInline represents the number of traces in the Inline direction of a complete window, and OverlapInline represents the number of overlapping traces in the Inline direction of a single overlapping window; the number of blocks nCrossline in the Crossline direction is calculated as follows: nCrossline=(CrossN-OverlapCrossline) / (WinCrossline– OverlapCrossline)+1 where CrossN represents the total number of lines in the Crossline direction, WinCrossline represents the number of traces in the Crossline direction of a complete window, and OverlapCrossline represents the number of overlapping traces in the Crossline direction of a single overlapping window; the product of the number of blocks nInline and the number of blocks nCrossline is the number of non-overlapping windows; before determining whether it is an overlapping trace, the trace is marked as follows: for each trace data, first mark flagInline = 0, flagCrossline = 0; then calculate the window position (iInline, iCrossline) in which the trace data is located; when iInline >=1, iCrossline >=1, determine whether the trace is an overlapping trace: if InlineNum <= (iInline -1)*(WinInline - OverlapInline)+WinInline -1, mark flagInline = -1 in the Inline direction; if CrosslineNum<=(iCrossline-1)*(WinCrossline- OverlapCrossline) +WinCrossline -1, mark flagCrossline = -1 in the Crossline direction; determine whether it is an overlapping trace and the non-overlapping window to which it belongs according to the mark If flagInline + flagCrossline == 0, then flagInline = 0, flagCrossline = 0, which means that the trace data is not in the overlapping area; Otherwise, if flagInline * flagCrossline == 0, then flagInline = 0, flagCrossline = -1, which means that the trace data is in the Crossline direction overlapping area; flagInline = -1, flagCrossline = 0, which means that the trace data is in the Inline direction overlapping area, so the Inline direction overlapping area trace data also belongs to the window: (iInline + flagInline) * nCrossline + (iCrossline + flagCrossline) ; If flagInline * flagCrossline!= 0, then flagInline = -1, flagCrossline = -1, which means that the trace data is in the overlapping area of two directions, so the overlapping area also belongs to the window: (iInline + flagInline) * nCrossline + (iCrossline + flagCrossline), (iInline + 0) * nCrossline + (iCrossline + flagCrossline), (iInline + flagInline) * nCrossline + (iCrossline + 0).

2. The parallel grain partitioning method for Spark-based seismic data according to claim 1, wherein, The method for constructing the trace data RDD comprises the following steps: Input the seismic data into the distributed file system for partitioning, each partition corresponds to the data of multiple lines, and the data of each line is composed of multiple trace data; According to the data in the partition, an Inline number RDD is constructed, and each element in the Inline number RDD corresponds to a key value pair represented as , The value is InlineNum, that is, the line number in the Inline direction, The value is all track data of the co-Inline number; The Inline number RDD is converted into a trace data RDD, and each element in the trace data RDD is represented as a key-value pair , The value of the key is (InlineNum, CrosslineNum), and the value of the value is corresponding trace data, InlineNum represents the line number in the Inline direction, and CrosslineNum represents the line number in the Crossline direction.​ 3. The Spark-based parallel grain partitioning method for seismic data of claim 2, using the flatMap transformation function to extract the Inline number and Crossline number from the trace header as the values of for each trace data in the Inline number RDD. ​ 4. The parallel grain partitioning method for Spark-based seismic data according to claim 3, wherein, The window position of the non-overlapping window is determined by the window position iInline in the Inline direction and the window position icrossline in the Crossline direction, and iInline and icrossline are numbered from 0.

5. The parallel granularity division method for Spark-based seismic data according to claim 4, characterized in that, The key-value pair corresponding to each element in the window form RDD is expressed as , The value is the window number, The value is all channel data within the complete window corresponding to . Calculated as follows: In the formula, represents the window position in the Inline direction, represents the window position in the Crossline direction, represents the number of blocks in the Crossline direction.

6. The application of a parallel grain partitioning method for seismic data based on Spark according to any one of claims 1-5, characterized in that, The method is used for dividing the pre-stack seismic data, post-stack seismic data, three-dimensional seismic data or two-dimensional seismic data.

7. The application of a Spark-based parallel grain partitioning method for seismic data according to claim 6, characterized in that, After the data processing according to the parallel granularity is completed, if it is required to output according to the Inline number RDD, the window RDD is first converted into the trace data RDD with InlineNum and CrosslineNum as the key values, the data is de-duplicated, and then the Inline is converted into the RDD with Inline as the key value.

Images