Parallel seismic data screening device and method
By utilizing the elastic distributed RDD dataset of Spark's parallel data model and employing input parameter acquisition, parameter transformation, and output control modules, the problem of non-universal seismic data filtering was solved, enabling fast and accurate data filtering and improving the efficiency of seismic data processing.
Patent Information
- Application Number
- CN202411653711.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-19
- Publication Date
- 2026-05-19
AI Technical Summary
Existing methods for filtering and processing earthquake data are not universal and cannot quickly meet various filtering conditions, resulting in low operational efficiency.
Using the Spark parallel data model and the elastic distributed RDD dataset, the system filters out seismic data that meets and does not meet the criteria through input parameter acquisition, parameter transformation, RDD dataset processing, and output control modules, and sets the output ports accordingly.
It enables rapid filtering of seismic data based on user-defined criteria, improving the efficiency of seismic data processing and ensuring the accuracy and efficiency of data filtering operations.
Smart Images
Figure CN122063669A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of seismic exploration, specifically relating to a parallel seismic data screening device and method. Background Technology
[0002] In seismic exploration, it is sometimes necessary to filter seismic data to obtain the gathers that need to be processed. Currently, the general method for filtering seismic data is to filter the entire input gathers according to specific requirements. However, this method is not universal and cannot meet the need to quickly select the required seismic data under various filtering conditions, resulting in relatively low operational efficiency. Summary of the Invention
[0003] The purpose of this invention is to solve the problems existing in the prior art. In view of the shortcomings of the prior art, this invention utilizes the data transformation characteristics of Spark's parallel data model, elastic distributed RDD, to develop a parallel seismic data screening device.
[0004] First, acquire data from two elastic distributed RDD datasets. Filter out data from one dataset according to given parameters, removing those that do not meet the criteria. Filter out data from the other dataset according to given parameters, removing datasets with zero trace counts from both the matching and non-matching datasets, leaving only trace set elements with non-zero trace counts. Finally, control the data output by setting two output ports to output the matching and non-matching trace sets separately.
[0005] This invention is achieved through the following technical solution:
[0006] In a first aspect, the present invention provides a parallel seismic data filtering device, the device comprising:
[0007] Input parameter acquisition module: used to acquire the filtering condition parameters of the keyword input by the user through the user interface;
[0008] Parameter Conversion Module: This module is used to create relevant parameters for the search field based on the filter condition parameters obtained by the input parameter acquisition module, and to convert the filter condition parameters from external values to the filter condition values required for internal calculation.
[0009] RDD Dataset Processing Module: Used to acquire Resilient Distributed Datasets (RDDs) from external systems;
[0010] The filtering module is used to filter the Trace data that meets the filtering criteria and the Trace data that does not meet the filtering criteria based on the filtering criteria values obtained from the parameter transformation module, and to obtain two sets of filtered Elastic Distributed RDD Trace data.
[0011] Output control module: Set port outputs for the two filtered elastic distributed RDD collections respectively.
[0012] Furthermore, the input parameter acquisition module obtains all the filter condition parameters input by the user by using the parameter ID to obtain the parameter object.
[0013] Furthermore, the parameter conversion module processing flow includes:
[0014] Step 1: Use the tolerance control of keyword value to realize a set of filtering condition parameters within a certain range as the value of the same target for identification processing;
[0015] Step 2: Use the convertFilter function to convert the filter condition parameters into internal values.
[0016] Furthermore, in step 1, tolerance control refers to: using the user-specified filtering condition parameter as a control point, expanding this value to a range through tolerance data, and treating all values within this range as control points.
[0017] Furthermore, in step 1, the tolerance control uses a specific algorithm to process the tolerance range of data values; the specific algorithm, based on the control points specified by the user, identifies values within a certain range according to the tolerance data and uses them as valid data.
[0018] Furthermore, step 2 includes:
[0019] The first step is to create a FileFilter object using the getFileFilter function and add all the necessary filter parameters;
[0020] The second step is to use the convertFilter function to convert the values of the filter parameters from user-defined external values to values used internally by the system.
[0021] Furthermore, the elastic distributed RDD input by the external system has an element that is a tuple in the form of (TraceHeaderKey, Record), where TraceHeaderKey is an object of the TraceHeaderKey encapsulation class and Record is an object of the TraceSet encapsulation class; the elastic distributed RDD is in two copies: one copy is used to filter TraceSets that meet the filtering criteria, and the other copy is used to filter TraceSets that do not meet the filtering criteria.
[0022] Furthermore, the filtering module uses the filterRecord and filterOtherRecord functions to filter out the Trace sets that meet the filtering criteria and those that do not. For the two elastic distributed RDDs, the map function is used to apply the filterRecord and filterOtherRecord functions to filter the Trace sets. The filtered records are still returned in the form of RDDs.
[0023] Furthermore, the filtering module also uses the filter function in the Scala language to remove trace sets with a trace count of zero.
[0024] A second aspect of the present invention provides a parallel seismic data filtering method, wherein the parallel seismic data filtering device is used to filter seismic data, characterized in that: the method includes:
[0025] Step 1: Obtain the data of the elastic distributed RDD from the external system through the RDD dataset processing module;
[0026] Step 2: Obtain all user-defined filter parameters through the input parameter acquisition module;
[0027] Step 3: Through the parameter conversion module, the filtering condition parameters obtained by the input parameter acquisition module are converted from external values into the filtering condition values required for internal calculation.
[0028] Step 4: Through the filtering processing module, based on the filtering condition values obtained from the parameter transformation module, two filtering functions are used to filter the Trace data that meets the filtering conditions and the Trace data that does not meet the filtering conditions, and finally two filtered elastic distributed RDDs are obtained respectively.
[0029] Step 5: Through the output control module, set the port output for the two filtered elastic distributed RDDs respectively: the right port outputs the collections that meet the filtering conditions, and the left port outputs the collections that do not meet the filtering conditions.
[0030] Compared with the prior art, the beneficial effects of the present invention are:
[0031] Users of the module only need to enter the required filtering conditions in the parameter conditions to quickly filter out the gathers that meet the filtering conditions and remove the gathers that do not meet the filtering conditions, thereby improving the efficiency of seismic data processing. Attached Figure Description
[0032] Figure 1 The overall flowchart of the screening method of this invention.
[0033] Figure 2 A parameter panel diagram of the screening device of the present invention.
[0034] Figure 3 The processing flowchart of the screening module of this invention. Detailed Implementation
[0035] This invention utilizes Spark parallel processing technology to quickly filter out Trace sets that meet and do not meet the input filtering criteria. Users only need to input user-defined keyword filtering conditions, which are then output through the platform's output module or passed to other modules for use.
[0036] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0037] Example 1: Parallel Seismic Data Filtering Device
[0038] The parallel seismic data filtering device of the present invention includes the following working modules:
[0039] 1. Input Parameter Acquisition Module: Users input custom filtering conditions for track heads through the user interface. The input parameter acquisition module retrieves all custom filtering conditions. Specifically, it retrieves all user-input filtering conditions by using the parameter ID to obtain the parameter object, which is then used by the algorithm to filter track sets based on the conditions.
[0040] 2. Parameter Conversion Module: Based on the user-input condition parameters, create relevant parameters for the search field, such as start value, end value, increment, etc.; and convert the user-input condition parameters from external values into the filter condition values required for internal calculation.
[0041] 3. RDD Dataset Processing Module: Obtains elastic distributed RDDs from external systems and prepares them for filtering operations.
[0042] The external system and the parallel seismic data filtering device of this invention are applications of the πFrame platform dedicated to seismic processing. In the Spark workflow, the external system inputs elastic distributed RDD data, which is directly obtained through the elastic distributed RDD data acquisition method set in this module.
[0043] This module provides the filtering processing module with an Elastic Distributed RDD dataset for filtering operations. Specifically, it obtains two Elastic Distributed RDD datasets from the external system: one dataset is used to filter Trace data that meets the criteria, and the other dataset is used to filter Trace data that does not meet the criteria.
[0044] 4. Filtering Module: Based on the filtering criteria values obtained from the parameter transformation module, two filtering functions are used to filter the trace data that meets the filtering criteria and the trace data that does not meet the filtering criteria, respectively, ultimately obtaining two sets of filtered RDD trace data. One set contains trace data that meets the filtering criteria; the other set contains trace data that does not meet the filtering criteria; and trace sets with a trace count of zero are filtered out.
[0045] 5. Output Control Module: This module sets separate port outputs for the filtering results. The right port outputs the trace sets that meet the filtering criteria, while the left port outputs the trace sets that do not. Users can configure the module's front-end interface to retrieve either both trace set data simultaneously or only a single trace set.
[0046] The entire process of the device is a continuous process from user input to data processing and then to result output. Each module plays a specific role to ensure the accuracy and efficiency of data filtering operations.
[0047] The specific technical methods used to implement each module are as follows:
[0048] 1. Input Parameter Acquisition Module: Users can input custom keyword filtering conditions through the user interface.
[0049] Users obtain custom keyword filtering criteria parameters for the search engine through the input parameter acquisition module. The filtering criteria support combined searches of multiple keywords, and logical relationships such as AND and OR can be specified between fields. The definition of the search fields includes the start value, end value, increment, absolute value, tolerance, control over whether to output filtering results, and the combination relationship of AND / OR conditions.
[0050] Retrieves all user-defined keyword filtering condition parameters for track sets. Specifically, a library function is used to retrieve parameter objects based on parameter IDs. The parameter interface is generated by parsing a JSON file, and each parameter has an ID name. The library function can directly read the ID name to obtain the parameter, thus retrieving all user-input filtering condition parameters for the filtering processing module to use when filtering track sets based on conditions.
[0051] 2. Parameter Conversion Module
[0052] like Figure 2 As shown, based on the user input parameters obtained from the user input module, relevant parameters for the search field are created, such as start value, end value, and increment; and the user input parameters are converted from external values to the filtering condition values required for internal calculation. The functional implementation process of this module includes:
[0053] (1) A set of values within a certain range refers to the values of the screening condition parameters as the values of the same target for identification and processing.
[0054] It supports tolerance control for keyword values, enabling a set of values (i.e., the values of filter criteria parameters) within a certain range to be treated as values for the same target and processed accordingly. Specifically, this includes a tolerance function that requires the combined use of tolerance control and data range control. That is, a series of user-specified values (filter criteria parameters) serve as control points, providing a tolerance data. Specifically, a control point is a specific numerical value, and the tolerance expands this value into a range. Values within this range can be considered as control points and processed accordingly. The range of values (closed-before-open interval) is then defined as [control point - tolerance, control point + tolerance], for example, in the form [a, b). Other forms are also acceptable, as long as they do not intersect with other intervals, they are considered as control points for filtering.
[0055] Furthermore, to achieve tolerance control, this invention develops a specific algorithm to handle the tolerance range of data values. Based on user-specified control points and combined with tolerance data, the algorithm identifies values within a certain range and filters them as valid data.
[0056] Preferably, for the tolerance control algorithm of seismic data screening operation, this invention can use a deep learning neural network based on fractional-order optimal control theory. This method explains the principle of the network from the perspective of optimal control of dynamic systems, and enhances the stability of the network by using long-term memory, thereby improving the system's ability to reduce noise.
[0057] (2) Obtain all conditional parameters and convert them to internal values.
[0058] The first step is to create a FileFilter object using the getFileFilter function and add all the necessary conditional parameters.
[0059] Specifically, this includes the file filtering class `FileFilter` and the sample class `FilterItem`. The `FileFilter` class contains a parameter storage container, parameter addition functions, parameter retrieval functions, and filtering functions to determine if a parameter meets certain conditions. A `FileFilter` object with all conditional parameters is obtained by calling the `getFileFilter` function. The `getFileFilter` function calls the functions in the `FileFilter` class for retrieving and adding conditional parameters. The `FileFilter` object returned by the `getFileFilter` function contains all the conditional parameters.
[0060] The second step is to use the convertFilter function to convert the values of these parameters from user-defined external values to values used internally by the system.
[0061] Use the filtering functions of the FileFilter object to perform filtering operations on the data.
[0062] The FilterItem class is the defined condition parameter class, containing all fields; each FilterItem object corresponds to a row of condition parameters; the parameters obtained in the getFileFilter function are individual FilterItem objects.
[0063] Preferably, the getFileFilter function is a function in the module. This function is mainly used to obtain an object containing all the parameter conditions, add all the condition parameters to the FileFilter object and return it, and then perform further processing.
[0064] 3. RDD Dataset Processing Module
[0065] This module obtains the Elastic Distributed RDD dataset input from the external system according to processing requirements. The RDD collection data elements of this dataset are a tuple in the form of (TraceHeaderKey, Record). The first element of the tuple is an object of the TraceHeaderKey wrapper class, and the second element is an object of the collection wrapper class. Two Elastic Distributed RDD datasets are obtained here for subsequent filtering: one is used to filter collections that meet the filtering criteria, and the other is used to filter collections that do not meet the filtering criteria.
[0066] Specifically, the data file input from the external system is read using the `textFile` method of `SparkContext`, creating a Resilient Distributed Dataset (RDD). This involves parsing each record, extracting the header key and record data, and encapsulating them into corresponding objects.
[0067] Preferably, if the data output by the previous module is not directly in the form of binary tuples, some data cleaning and transformation work is required, specifically through transformation operations such as map, flatMap, or mapPartitions. For example, flatMap can be used when it is necessary to split a record into multiple records; while mapPartitions can be used when it is necessary to batch process the data within a partition.
[0068] 4. Filtering module
[0069] This module filters data based on the elastic distributed RDD collections provided by the RDD dataset processing module. The `filterRecord` and `filterOtherRecord` functions are used to filter collections that meet the criteria and those that do not. The built-in Spark operator `map` is used, with the function passed directly as an argument. The `map` function is then called using the retrieved RDD collections to achieve the data filtering objective. The processing steps include:
[0070] (1) Obtaining a resilient distributed RDD dataset
[0071] Two resilient distributed RDD datasets are obtained from the RDD dataset processing module. Each resilient distributed RDD dataset contains seismic trace data, and the data elements are in the format of tuples (TraceHeaderKey, Record).
[0072] (2) Define the filtering function
[0073] The algorithm uses two functions defined in the algorithm to filter the record set, namely filterRecord and filterOtherRecord.
[0074] The FilterRecord function filters out tracks that do not meet the given conditions according to the given parameters. This function is used to filter out a set of tracks that meet specific conditions. It checks whether the header information (TraceHeaderKey) of each track meets the conditions defined in the FileFilter object.
[0075] The filterOtherRecord function is used to filter out records that do not meet specific conditions; its logic is the opposite of filterRecord.
[0076] (3) Use the map function for filtering
[0077] For two resilient distributed RDD datasets, the map function is used to apply the filterRecord and filterOtherRecord functions respectively to perform trace set filtering. These functions take a (TraceHeaderKey, Record) tuple and a FileFilter object as parameters.
[0078] Inside the filterRecord and filterOtherRecord functions, the filter member function of the FileFilter object is called to determine whether the record header meets the conditions.
[0079] (4) Record the screening results
[0080] The filterRecord function records the records that meet the filtering criteria in Record1; the filterOtherRecord function records the records that do not meet the filtering criteria in Record2.
[0081] The filtered records are still returned as an RDD.
[0082] (5) Filtering gathers with zero channel count
[0083] The results are further processed using the `filter` function in Scala to remove channels with a channel count of zero. This ensures that the results do not contain empty or invalid datasets.
[0084] After the above steps, two filtered Resilient Distributed RDD datasets are finally obtained: one containing Tales sets that meet the filtering criteria, and the other containing Tales sets that do not meet the filtering criteria.
[0085] Figure 3 The process involves processing input RDD collection data through a filtering module and outputting datasets that meet the filtering criteria and datasets that do not meet the filtering criteria.
[0086] 5. Output control module
[0087] The filtered elastic distributed RDD datasets that meet the filtering criteria and those that do not are separated by an output function, and the results are output.
[0088] Specifically, after filtering, two RDD datasets will be obtained: SeisData, which contains seismic data that meets the filtering criteria; and otherSeisData, which contains seismic data that does not meet the filtering criteria.
[0089] The custom functions `this.setOutputRDD(SeisData, 0)` and `this.setOutputRDD(otherSeisData, 1)` are output library functions that control the output ports: output datasets that meet the filtering criteria are output through the left port, and output datasets that do not meet the filtering criteria are output through the right port. Specifically, each dataset is identified by a different port.
[0090] this.setOutputRDD(SeisData,0): Sets the seismic dataset SeisData that meets the filtering criteria as the output, and identifies this output port by the parameter 0.
[0091] this.setOutputRDD(otherSeisData,1): Sets the seismic dataset otherSeisData that does not meet the filtering criteria as the output, and identifies this output port by the parameter 1.
[0092] Here, parameters 0 and 1 serve as port identifiers to distinguish different output streams.
[0093] Furthermore, in actual Spark applications, this corresponds to different file paths, database tables, or output queues, etc.
[0094] Preferably, users can choose to output two datasets simultaneously, or only output datasets that meet or do not meet the filtering criteria, as needed.
[0095] Example 2: Parallel Seismic Data Filtering Method
[0096] The method uses a parallel seismic data filtering device as described above, and filters seismic data through the following steps:
[0097] Step 1, acquire earthquake data
[0098] The RDD dataset processing module obtains the TraceHeaderKey data from the external system's Elastic Distributed RDD. Each TraceHeaderKey data element in this dataset is a tuple of the form (TraceHeaderKey, Record). The first element of this tuple is an object of the TraceHeaderKey wrapper class, and the second element is an object of the TraceHeaderKey wrapper class. This step obtains two Elastic Distributed RDD datasets for subsequent filtering: one for filtering TraceHeaders that meet the filtering criteria, and the other for filtering TraceHeaders that do not meet the criteria.
[0099] Step 2, Obtain Condition Parameters
[0100] The input parameter acquisition module obtains all custom filter condition parameters. Specifically, it obtains all user-input filter condition parameters by retrieving parameter objects based on parameter IDs, which are then used by the algorithm to filter the track set based on the conditions.
[0101] Step 3: Convert the conditional parameter values into internal values.
[0102] The parameter conversion module obtains all user-defined filter condition parameters from the input parameter acquisition module, creates relevant parameters for the search field, and converts the user-input condition parameters from external values into the filter condition values required for internal calculation.
[0103] Step 4, Filter the dataset
[0104] In the filtering module, based on the filtering criteria values obtained from the parameter transformation module, two filtering functions filter the trace data that meets the filtering criteria and the trace data that does not meet the filtering criteria, respectively, ultimately obtaining two sets of filtered RDD trace data. One set contains the trace data that meets the filtering criteria; the other set contains the trace data that does not meet the filtering criteria, and trace sets with a trace count of zero are filtered out.
[0105] Step 5, Output the dataset
[0106] The output control module allows users to configure output ports for each filtering result. The right port outputs the trace sets that meet the filtering criteria, while the left port outputs the trace sets that do not meet the filtering criteria. Users can choose to acquire data from both trace sets simultaneously or only from a single trace set.
[0107] The above technical solution is only one embodiment of the present invention. For those skilled in the art, based on the principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the technical solutions described in the specific embodiments of the present invention. Therefore, the foregoing description is only a preferred option and is not restrictive.
Claims
1. A parallel seismic data filtering device, characterized in that: The device includes: Input parameter acquisition module: used to acquire the filtering condition parameters of the keyword input by the user through the user interface; Parameter Conversion Module: This module is used to create relevant parameters for the search field based on the filter condition parameters obtained by the input parameter acquisition module, and to convert the filter condition parameters from external values to the filter condition values required for internal calculation. RDD Dataset Processing Module: Used to acquire Resilient Distributed Datasets (RDDs) from external systems; The filtering module is used to filter the Trace data that meets the filtering criteria and the Trace data that does not meet the filtering criteria based on the filtering criteria values obtained from the parameter transformation module, and to obtain two sets of filtered Elastic Distributed RDD Trace data. Output control module: Set port outputs for the two filtered elastic distributed RDD collections respectively.
2. The parallel seismic data filtering device according to claim 1, characterized in that: The input parameter acquisition module uses the parameter ID to obtain parameter objects and retrieves all the filter condition parameters input by the user.
3. The parallel seismic data filtering device according to claim 1, characterized in that: The processing flow of the parameter conversion module includes: Step 1: Use the tolerance control of keyword value to realize a set of filtering condition parameters within a certain range as the value of the same target for identification processing; Step 2: Use the convertFilter function to convert the filter condition parameters into internal values.
4. The parallel seismic data filtering device according to claim 3, characterized in that: In step 1, tolerance control means: using the user-specified filtering condition parameter as a control point, expanding this value to a range through tolerance data, and treating all values within this range as control points.
5. The parallel seismic data filtering device according to claim 3, characterized in that: In step 1, tolerance control uses a specific algorithm to process the tolerance range of data values. Based on the control points specified by the user, the specific algorithm identifies values within a certain range according to the tolerance data and uses them as valid data.
6. The parallel seismic data filtering device according to claim 3, characterized in that: Step 2 includes: The first step is to create a FileFilter object using the getFileFilter function and add all the necessary filter parameters; The second step is to use the convertFilter function to convert the values of the filter parameters from user-defined external values to values used internally by the system.
7. The parallel seismic data filtering device according to claim 1, characterized in that: The elastic distributed RDD input from the external system has an element that is a tuple in the form of (TraceHeaderKey, Record), where TraceHeaderKey is an object of the TraceHeaderKey wrapper class and Record is an object of the TraceSet wrapper class. The elastic distributed RDD is divided into two copies: one copy is used to filter TraceSets that meet the filtering criteria, and the other copy is used to filter TraceSets that do not meet the filtering criteria.
8. The parallel seismic data filtering device according to claim 1, characterized in that: The filtering module uses the filterRecord and filterOtherRecord functions to filter out the Trace sets that meet the filtering criteria and those that do not. For the two elastic distributed RDDs, the map function is used to apply the filterRecord and filterOtherRecord functions to filter the Trace sets. The filtered records are still returned in the form of RDDs.
9. The parallel seismic data filtering device according to claim 1, characterized in that: The filtering module also uses the filter function in the Scala language to remove trace sets with a trace count of zero.
10. A parallel seismic data filtering method, using the parallel seismic data filtering device according to any one of claims 1-9 to filter seismic data, characterized in that: The method includes: Step 1: Obtain the data of the elastic distributed RDD from the external system through the RDD dataset processing module; Step 2: Obtain all user-defined filter parameters through the input parameter acquisition module; Step 3: Through the parameter conversion module, the filtering condition parameters obtained by the input parameter acquisition module are converted from external values into the filtering condition values required for internal calculation. Step 4: Through the filtering processing module, based on the filtering condition values obtained from the parameter transformation module, two filtering functions are used to filter the Trace data that meets the filtering conditions and the Trace data that does not meet the filtering conditions, and finally two filtered elastic distributed RDDs are obtained respectively. Step 5: Through the output control module, set the port output for the two filtered elastic distributed RDDs respectively: the right port outputs the collections that meet the filtering conditions, and the left port outputs the collections that do not meet the filtering conditions.