A data preparation method and device for big data analysis of fracturing effect

Abstract

The present application relates to the technical field of oilfield production engineering, and particularly relates to a data preparation method and device for big data analysis of fracturing effect. The method mainly solves the problems of complex data preparation work and high time proportion in the process of big data analysis of fracturing effect. The method comprises the following steps: collecting parameter data related to fracturing of a target block to generate a source data set; generating a fracturing information data set based on the source data set, wherein the fracturing information data set comprises a basic parameter set and a derived data set; sequentially performing data cleaning, data standardization, correlation analysis and feature parameter screening on the data in the fracturing information data set to generate an influence factor analysis table. The data preparation method for big data analysis of fracturing effect can quickly generate data that can be directly used for fracturing effect analysis and fracturing scheme optimization model, and the generated data has reliability, correlation and interpretability, thereby providing high-quality input for subsequent fracturing effect analysis model.

Description

Technical Field

[0001] This invention relates to the field of oilfield data analysis and processing, specifically to a data preparation method and apparatus for big data analysis of fracturing effects. Background Technology

[0002] With the development of oilfield digitalization, the volume of data is increasing, and the types of data available are becoming more diverse, providing possibilities for big data mining to optimize fracturing schemes and improve development results. Faced with massive amounts of data, high-quality data preparation is the foundation and key to improving model computation efficiency and prediction accuracy, often accounting for 60% to 90% of the workload in a complete data mining process. The amount of data related to fracturing engineering in oilfields is large, and the extraction and integration of raw data, the cleaning and standardization of "dirty" data, and the definition and screening of characteristic parameters are all challenges affecting the efficiency and effectiveness of data mining.

[0003] Currently published common data preparation methods mainly involve routine classification and screening of acquired big data, followed by feature selection using correlation analysis to achieve data preprocessing. For example, in patent CN115310357A, reservoir parameters, rock mechanics parameters, fracturing operation parameters, and production capacity parameters were optimized and screened for acquired oilfield data, and Pearson correlation analysis was used to complete the data preprocessing. However, these methods have limitations in data selection and extraction, ignoring some potentially significant derivative data that may affect fracturing results, and they do not fundamentally provide efficient methods for handling data anomalies, missing data, and feature selection. Summary of the Invention

[0004] To overcome the shortcomings of existing big data analysis of fracturing effects, which involves complex data preparation and high time consumption, this invention provides a data preparation method and apparatus for big data analysis of fracturing effects. This data preparation method can quickly generate data that can be directly used for fracturing effect analysis and fracturing scheme optimization models. The generated data is reliable, relevant, and interpretable, providing high-quality input for subsequent fracturing effect analysis models.

[0005] The technical solution of this invention is: a data preparation method for big data analysis of fracturing effects, comprising the following steps:

[0006] S1. Collect parameter data related to fracturing in the target block and generate source dataset;

[0007] S2. Generate a fracturing information dataset based on the source dataset;

[0008] The fracturing information dataset includes a basic parameter set and a derived dataset. The basic parameter set is obtained by extracting corresponding parameters from the source dataset based on different data, or by using a corresponding derived parameter algorithm to calculate the extracted parameters. The derived dataset is obtained by extracting relevant parameters from the source dataset or the basic parameter set based on different data, or by using a corresponding derived parameter algorithm to calculate the extracted parameters.

[0009] S3. Perform data cleaning, data standardization, correlation analysis, and feature parameter screening on the data in the fracturing information dataset in sequence to generate an influencing factor analysis table.

[0010] Furthermore, step S3 also includes converting the parameters in the selected fracturing information dataset into visual graphics and generating a preliminary descriptive data analysis report.

[0011] Furthermore, the source dataset includes: a comprehensive oil well production data table, a single well perforation section data table, a monthly oil well production data table, a monthly water well production data table, a detailed information table of single well fracturing construction procedures, a single-layer fracturing information table, and a single well location information table.

[0012] Furthermore, the basic parameter set includes a well basic information database, a geological parameter table, a measure parameter table, a measure procedure table, an oil well production dynamic table, and a water well production dynamic table; the derived dataset includes a table of influencing factors for single-well measures in oil wells, a table of influencing factors for single-layer measures in oil wells, a table of influencing factors for single-well measures in water wells, and a table of influencing factors for single-layer measures in water wells.

[0013] Furthermore, the data cleaning in step S3 involves performing a missing value statistical analysis on the data. If the missing value of a certain parameter is greater than a first predetermined value, the parameter is deleted; if the missing value is not greater than the first predetermined value, the data is supplemented.

[0014] Furthermore, the data supplement includes:

[0015] Identify the missing basic parameter categories, which include geological parameters, engineering parameters, and effect parameters;

[0016] Cluster all parameters in the category of the missing parameter to divide the data into several different clusters, and use non-empty parameters in the same cluster to fill in the missing parameter.

[0017] Furthermore, when supplementing missing parameters with non-empty parameters from the same cluster, the mean is used to fill in numerical data, and the mode is used to fill in character data.

[0018] Furthermore, the first predetermined value is 30%.

[0019] Furthermore, the data standardization in step S3 involves converting the cleaned character data into numerical data and then normalizing the data.

[0020] Furthermore, the correlation analysis in step S3 involves using the Pearson correlation coefficient to remove data with a correlation coefficient higher than a second predetermined value, and using analysis of variance to remove data with a threshold lower than a third predetermined value.

[0021] Furthermore, the second predetermined value is 0.8, and the third predetermined value is 0.005.

[0022] Furthermore, in step S3, the feature parameter selection involves applying recursive gradient descent, Spearman correlation coefficient, decision tree, random forest, and machine learning methods to select feature parameters. The top ten feature parameters selected by each method are combined to obtain the optimal feature subset.

[0023] Furthermore, the visual graphics in step S3 include box plots, heatmaps, scatter plots, bar charts, radar charts, bubble charts, or clustering diagrams.

[0024] A data preparation device for big data analysis of fracturing effects includes: a data acquisition module, a data extraction module, a data visualization module, and a data governance module;

[0025] The data acquisition module is used to collect parameter data related to fracturing in the target block and generate a source dataset.

[0026] The data extraction module generates a fracturing information dataset based on the source dataset and by applying a derived parameter algorithm.

[0027] The data visualization module, based on the fracturing information dataset, converts the selected parameters into visual graphics and generates a preliminary descriptive data analysis report.

[0028] The data governance module performs data cleaning, data standardization, correlation analysis, and feature parameter screening on the centralized data of the fracturing information dataset in sequence, and generates an influencing factor analysis table.

[0029] This invention offers the following advantages: By employing the above-mentioned approach, through data acquisition, feature extraction, data governance, and visualization-assisted analysis, it can rapidly generate data that can be directly used for fracturing effect analysis and fracturing scheme optimization models. The proposed derived parameters, which are of practical significance for fracturing effect analysis, deeply mine the potential of the data, providing more and more useful information for fracturing effect analysis. The final generated data possesses reliability, relevance, and interpretability, providing high-quality input for subsequent fracturing effect analysis models. This reduces computational complexity, lowers the amount of computation, and improves model computational efficiency, prediction accuracy, and interpretability, laying a solid foundation for big data analysis of fracturing effects, optimization of fracturing schemes, and improvement of fracturing outcomes. Attached Figure Description

[0030] Figure 1 This is a flowchart of the data preparation method in this invention;

[0031] Figure 2 This is a schematic diagram of the data preparation device in this invention. Detailed Implementation

[0032] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. The technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Depend on Figure 1 As shown, a data preparation method for big data analysis of fracturing effects includes the following steps:

[0034] S1. Based on the oilfield production management data platform database, a data acquisition program is used to collect parameter data related to fracturing in the target block, and source datasets are generated according to different data sources to facilitate later source tracing.

[0035] The source dataset includes seven data tables: oil well production comprehensive data table, single well perforation section data table, oil well production monthly data table, water well production monthly data table, single well fracturing construction process detailed information table, single layer fracturing information table, and single well location information table.

[0036] The data acquisition program can extract data from the data lake and merge it to generate a new data table.

[0037] S2. Based on the source dataset, select all parameters that are relevant or potentially relevant to the entire fracturing project according to industry standards and the latest literature, and generate a fracturing information dataset.

[0038] The fracturing information dataset includes a basic parameter set and a derived dataset. The basic parameter set is obtained by extracting corresponding parameters from the source dataset based on different data, or by calculating the extracted parameters using a corresponding derived parameter algorithm. The derived dataset is obtained by extracting relevant parameters from the source dataset or the basic parameter set based on different data, or by calculating the extracted parameters using a corresponding derived parameter algorithm. The derived parameter algorithm is used to implement data extraction, definition and calculation of derived parameters, and merging them to generate a new data table.

[0039] The basic parameter set is mainly divided into three categories: geological parameters, engineering parameters, and effect parameters, containing a total of 6 data tables: well foundation information database, geological parameter table, measure parameter table, measure procedure table, oil well production dynamics table, and water well production dynamics table. Table 1 shows the parameter forms for each table in the basic parameter set. Parameters in the forms shown can be added or deleted as needed.

[0040] Table 1 shows the parameter lists for each table in the basic parameter set.

[0041]

[0042]

[0043] The derived datasets are mainly divided into two categories: single-well parameters and single-layer parameters. They record new, practically meaningful derived data generated using specific derivation algorithms and contain four tables: Oil Well Single-Well Measures Influence Factors Table, Oil Well Single-Layer Measures Influence Factors Table, Water Well Single-Well Measures Influence Factors Table, and Water Well Single-Layer Measures Influence Factors Table. Table 2 shows the parameter tables for each table in each derived dataset. Parameters in the tables can be added or deleted as needed, allowing analysts to flexibly adjust the parameters.

[0044] Table 2 shows the parameter list for each table in the derived dataset.

[0045]

[0046]

[0047]

[0048] In this embodiment of the invention, the data in the basic dataset originates from the source dataset, and is either directly extracted or calculated using corresponding derived parameter algorithms to form the basic dataset. The data in the derived dataset originates from both the source and basic datasets, and is similarly either directly extracted or calculated using corresponding derived parameter algorithms to form the derived dataset. The sources and algorithms for some of the parameters are shown in Table 3. As can be seen from the table, some data can be directly extracted from the source dataset, while other parameters require derived calculations.

[0049] The following examples illustrate the sources and algorithms for individual parameters:

[0050] ① Taking the well base database in the basic dataset as an example, the vertical axis is directly extracted from the single well location information table in the source dataset.

[0051] ② The number of fracturing operations in the well base database of the basic dataset is obtained by extracting data from the detailed information table of single-well fracturing construction procedures in the source dataset, and by judging and calculating the historical number of fracturing operations in a single well based on different fracturing dates of the same well.

[0052] ③ The well section top depth in the measure parameter table of the basic dataset is directly extracted from the single-layer fracturing information table in the source dataset.

[0053] ④ The amount of sand added in the basic dataset's centralized measure parameter table is obtained by extracting data from the single-well fracturing construction procedure details table in the source dataset, and then calculating the amount of sand added for the same well, the same fracturing date, and the same construction layer.

[0054] ⑤ The stable discharge rate of the oil well single-well measures in the derived dataset is extracted from the measures procedure table in the basic dataset.

[0055] ⑥ The permeability variation coefficient in the table of factors affecting single-well measures in the derived dataset is obtained by extracting data from the perforated section data table of single wells in the source dataset and calculating the standard deviation of permeability divided by the average permeability.

[0056] ⑦ The rupture pressure in the table of influencing factors of single-layer measures for water wells in the derived dataset is directly extracted from the measure procedure table in the basic dataset.

[0057] ⑧ The validity period of the table of influencing factors of single-layer measures for water wells in the derived dataset is obtained by extracting data from the table of water well production dynamics in the basic dataset and calculating the number of effective days of injection after pressure when the daily injection volume is lower than the daily injection volume before pressure for three consecutive months.

[0058] Due to the large number of parameters, they will not be listed here. The algorithms for each parameter can be understood by those skilled in the art based on existing technology.

[0059] Table 3. Sources of some parameters in the fracturing information dataset

[0060]

[0061] S3. Perform data governance on the data in the fracturing information dataset to generate an analysis table of influencing factors that can be directly used for fracturing effect evaluation and fracturing scheme optimization. During this process, the selected parameters can be converted into visual graphics and a preliminary descriptive data analysis report can be generated to describe the governance process or results in words.

[0062] The data governance involves sequentially performing data cleaning, data standardization, correlation analysis, and feature parameter screening on the data in the fracturing information dataset, specifically including:

[0063] S301. Data Cleaning. Data cleaning includes missing value analysis and data augmentation. Missing value analysis involves statistically analyzing the missing value of the data according to industry standard values ​​or empirical values, while also using visual graphics to intuitively identify outliers and null values. The missing value is determined based on the proportion of outliers and null values ​​to all data. If the missing value of a parameter is greater than a first predetermined value, that parameter is deleted; if the missing value is not greater than the first predetermined value, data augmentation is performed.

[0064] The first predetermined value can be set to 30%.

[0065] The data augmentation process first determines the category of missing basic parameters, which are categorized into geological parameters, engineering parameters, and effect parameters. Then, all parameters within the category of the missing parameter are clustered, dividing the data into several distinct clusters. Missing parameters are then augmented using non-empty parameters from the same cluster. When augmenting missing parameters, mean imputation is used for numerical data, and mode imputation is used for character data.

[0066] S302. Data Standardization. Data standardization involves converting cleaned character data into numerical data and normalizing the data to transform data with different dimensions and value ranges into uniform data between 0 and 1, thereby improving data consistency and facilitating its application in data analysis.

[0067] S303. Correlation analysis. The correlation analysis involves using the Pearson correlation coefficient to remove data with a correlation coefficient higher than a second predetermined value, and using analysis of variance to remove data with a threshold lower than a third predetermined value.

[0068] The second predetermined value is 0.8, and the third predetermined value is 0.005. That is, the Pearson correlation coefficient is used to remove data with a correlation coefficient higher than 0.8, and the analysis of variance is used to remove data with a threshold lower than 0.005.

[0069] S304. Feature Parameter Screening. The feature parameter screening involves applying recursive gradient descent, Spearman correlation coefficient, decision tree, random forest, and machine learning methods to screen feature parameters. The top ten feature parameters with the highest weights selected by each method are combined to obtain the optimal feature subset, thereby improving the accuracy and interpretability of subsequent model training.

[0070] In the data governance process, this invention allows analysts to select parameters to be analyzed as needed, and use specialized charting tools to automatically generate various types of visual graphics from the data for intuitive observation and analysis of data patterns and their relationships. It also generates a preliminary descriptive data analysis report to provide a verbal description of the governance process or results, offering analysts the results and suggestions from computer analysis.

[0071] The visual graphics include box plots, heatmaps, scatter plots, bar charts, radar charts, bubble charts, or cluster plots, which can be selected as needed.

[0072] This invention also provides a data preparation device for big data analysis of fracturing effects, comprising: Figure 2 As shown, it includes a data acquisition module, a data extraction module, a visualization module, and a data governance module;

[0073] The data acquisition module is used to collect parameter data related to fracturing in the target block and generate a source dataset.

[0074] The data extraction module generates a fracturing information dataset based on the source dataset and by applying a derived parameter algorithm.

[0075] The data visualization module, based on the fracturing information dataset, converts the selected parameters into visual graphics and generates a preliminary descriptive data analysis report.

[0076] The data governance module performs data cleaning, data standardization, correlation analysis, and feature parameter screening on the centralized data of the fracturing information dataset in sequence, and generates an influencing factor analysis table.

[0077] This invention, through data acquisition, feature extraction, data governance, and visualization-assisted analysis, can rapidly generate data that can be directly used for fracturing effect analysis and fracturing scheme optimization models. The proposed derived parameters, which are of practical significance for fracturing effect analysis, deeply mine the potential of the data and provide more and more useful information for fracturing effect analysis. The final generated data possesses reliability, relevance, and interpretability, providing high-quality input for subsequent fracturing effect analysis models. This reduces computational complexity and workload, improves model computational efficiency, prediction accuracy, and interpretability, laying a solid foundation for big data analysis of fracturing effects, optimization of fracturing schemes, and improvement of fracturing outcomes.

[0078] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A data preparation method for big data analysis of fracturing effects, characterized in that... include: S1. Collect parameter data related to fracturing in the target block and generate source dataset; S2. Generate a fracturing information dataset based on the source dataset; The fracturing information dataset includes a basic parameter set and a derived dataset. The basic parameter set is obtained by extracting corresponding parameters from the source dataset based on different data, or by using a corresponding derived parameter algorithm to calculate the extracted parameters. The derived dataset is obtained by extracting relevant parameters from the source dataset or the basic parameter set based on different data, or by using a corresponding derived parameter algorithm to calculate the extracted parameters. S3. Perform data cleaning, data standardization, correlation analysis, and feature parameter screening on the data in the fracturing information dataset in sequence to generate an influencing factor analysis table.

2. The data preparation method for big data analysis of fracturing effects according to claim 1, characterized in that: Step S3 further includes converting the parameters in the selected fracturing information dataset into visual graphics and generating a preliminary descriptive data analysis report.

3. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The source dataset includes: comprehensive oil well production data table, single well perforation section data table, monthly oil well production data table, monthly water well production data table, detailed information table of single well fracturing construction procedures, single-layer fracturing information table, and single well location information table.

4. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The basic parameter set includes a well basic information database, geological parameter table, measure parameter table, measure procedure table, oil well production dynamic table, and water well production dynamic table; the derived dataset includes a table of influencing factors for single-well measures of oil wells, a table of influencing factors for single-layer measures of oil wells, a table of influencing factors for single-well measures of water wells, and a table of influencing factors for single-layer measures of water wells.

5. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The data cleaning in step S3 involves performing a statistical analysis of the missing data. If the missing value of a parameter is greater than a first predetermined value, the parameter is deleted. If the missing value is not greater than the first predetermined value, the data is supplemented.

6. The data preparation method for big data analysis of fracturing effects according to claim 5, characterized in that: The data supplement includes: Identify the missing basic parameter categories, which include geological parameters, engineering parameters, and effect parameters; Cluster all parameters in the category of the missing parameter to divide the data into several different clusters, and use non-empty parameters in the same cluster to fill in the missing parameter.

7. The data preparation method for big data analysis of fracturing effects according to claim 6, characterized in that: When supplementing missing parameters with non-empty parameters from the same cluster, mean-filling is used for numerical data and mode-filling is used for character data.

8. The data preparation method for big data analysis of fracturing effects according to claim 7, characterized in that: The first predetermined value is 30%.

9. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The data standardization in step S3 involves converting the cleaned character data into numerical data and then normalizing the data.

10. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The correlation analysis in step S3 involves using the Pearson correlation coefficient to remove data with a correlation coefficient higher than a second predetermined value, and using analysis of variance to remove data with a threshold lower than a third predetermined value.

11. The data preparation method for big data analysis of fracturing effects according to claim 10, characterized in that: The second predetermined value is 0.8, and the third predetermined value is 0.

005.

12. The data preparation method for big data analysis of fracturing effects according to claim 1 or 2, characterized in that: The feature parameter selection in step S3 involves applying recursive gradient descent, Spearman correlation coefficient, decision tree, random forest, and machine learning methods to select feature parameters. The top ten feature parameters selected by each method are combined to obtain the optimal feature subset.

13. The data preparation method for big data analysis of fracturing effects according to claim 2, characterized in that: The visual graphics in step S3 include box plots, heatmaps, scatter plots, bar charts, radar charts, bubble charts, or cluster plots.

14. A data preparation device for big data analysis of fracturing effects, characterized in that... include: Data acquisition module, data extraction module, data visualization module, and data governance module; The data acquisition module is used to collect parameter data related to fracturing in the target block and generate a source dataset. The data extraction module generates a fracturing information dataset based on the source dataset and by applying a derived parameter algorithm. The data visualization module, based on the fracturing information dataset, converts the selected parameters into visual graphics and generates a preliminary descriptive data analysis report. The data governance module performs data cleaning, data standardization, correlation analysis, and feature parameter screening on the centralized data of the fracturing information dataset in sequence, and generates an influencing factor analysis table.

Images