An oilfield logging data management method and system based on big data

Abstract

The application discloses an oilfield logging data management method and system based on big data, and the method comprises the following steps: obtaining oilfield logging packaging data; performing big data analysis on the oilfield logging packaging data to obtain splicing data, data attributes, data types and flow types; performing data compression on the splicing data and the data attributes to obtain attribute compression data; performing bit compression on the attribute compression data and the data types to obtain type compression data; and obtaining a data storage position according to the type compression data and the flow types. Through decoding of various oilfield logging packaging data formats and big data classification, the method guarantees the uniformity of data; through data compression and bit compression, the method improves the compression performance and read-write efficiency of oilfield logging data, and finally improves the efficiency of logging data management.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data management, and in particular to an oilfield logging data management method and system based on big data. BACKGROUND

[0002] With the continuous growth of global energy demand and the progress of exploration and development technology, logging data, as the key information in oil and gas exploration and development, is applied throughout the whole process of oil and gas field exploration and development, with wide application range and high frequency of use.

[0003] However, due to the non-uniformity of the oilfield logging data packaging format, it is not conducive to the management and analysis of big data, and at the same time, due to the poor compression storage performance of the oilfield logging data, it is not conducive to the storage and reading of big data, which ultimately leads to low efficiency of logging data management. SUMMARY

[0004] In order to solve the above problems existing in the prior art, the present application provides an oilfield logging data management method and system based on big data. The technical problems to be solved by the present application are realized by the following technical solutions:

[0005] An oilfield logging data management method based on big data, comprising:

[0006] obtaining oilfield logging packaging data;

[0007] performing big data analysis on the oilfield logging packaging data to obtain splicing data, data attributes, data types and flow types;

[0008] performing data compression on the splicing data and the data attributes to obtain attribute compression data;

[0009] performing bit compression on the attribute compression data and the data types to obtain type compression data;

[0010] obtaining a data storage location according to the type compression data and the flow types.

[0011] In one specific embodiment, the big data analysis on the oilfield logging packaging data to obtain splicing data, data attributes, data types and flow types comprises:

[0012] adopting a corresponding decoder according to the oilfield logging packaging data format to obtain original data;

[0013] adopting big data analysis on the original data to obtain data attributes and flow types;

[0014] adopting big data classification on the data attributes to obtain data types;

[0015] Splice the original data according to the data attribute to obtain spliced data.

[0016] In one embodiment, the data compression of the spliced data and the data attribute to obtain attribute compression data comprises:

[0017] According to the spliced data and the data attribute, a prediction operator, a transformation operator and an encoding operator are obtained.

[0018] According to the spliced data, the prediction operator, the transformation operator and the encoding operator, a data compression method is obtained.

[0019] The data compression method is used on the spliced data to obtain type compression data.

[0020] In one embodiment, the prediction operator, the transformation operator and the encoding operator are obtained according to the spliced data and the data attribute, comprising:

[0021] According to the spliced data, a data dimension is obtained.

[0022] According to the data dimension, a corresponding prediction method, a transformation method or an encoding method is obtained.

[0023] According to the data attribute and the data dimension, a prediction unit, a transformation unit and an encoding unit are obtained.

[0024] According to the prediction method and the prediction unit, a prediction operator is obtained.

[0025] According to the transformation method and the transformation unit, a transformation operator is obtained.

[0026] According to the encoding method and the encoding unit, an encoding operator is obtained.

[0027] In one embodiment, the data compression method is obtained according to the spliced data and the prediction operator, the transformation operator and the encoding operator:

[0028] Data analysis is used on the spliced data to obtain a data analysis result.

[0029] According to the data analysis result, a prediction type, a transformation type and an encoding type are obtained.

[0030] According to the prediction type and the prediction operator, a prediction compression method is obtained.

[0031] According to the transformation type and the transformation operator, a transformation compression method is obtained.

[0032] According to the encoding type and the encoding operator, an encoding compression method is obtained.

[0033] The data compression method is obtained according to the prediction compression method, the transform compression method and the encoding compression method.

[0034] In one embodiment, the bit compression is performed on the attribute compression data and the data type to obtain type compression data, including:

[0035] An index encoding operator and a bit encoding operator are obtained according to the data type;

[0036] A bit compression method is obtained according to the attribute compression data, the index encoding operator and the bit encoding operator;

[0037] The bit compression method is performed on the attribute compression data to obtain type compression data.

[0038] In one embodiment, the index encoding operator and the bit encoding operator are obtained according to the data type, including:

[0039] A data attribute number and a data attribute feature are obtained according to the data type;

[0040] An index encoding method and a bit encoding method are obtained according to the data attribute number and the data attribute feature;

[0041] A bit encoding unit is obtained according to the data type and the data attribute number;

[0042] An index encoding operator is obtained according to the index encoding method;

[0043] A bit encoding operator is obtained according to the bit encoding method and the bit encoding unit.

[0044] In one embodiment, the bit compression method is obtained according to the attribute compression data, the index encoding operator and the bit encoding operator, including:

[0045] A compression analysis result is obtained by performing compression analysis on the attribute compression data;

[0046] An index encoding type and a bit encoding type are obtained according to the compression analysis result;

[0047] The bit compression method is obtained according to the index encoding type, the index encoding operator, the bit encoding type and the bit encoding operator.

[0048] In one embodiment, the data storage location is obtained according to the type compression data and the traffic type, including:

[0049] A first-level index is obtained according to the traffic type;

[0050] According to the type compression data, secondary index is obtained;

[0051] According to the primary index and the secondary index, data storage location is obtained.

[0052] 10. A big data-based oilfield logging data management system, characterized in that it comprises:

[0053] An acquisition unit configured to acquire oilfield logging package data;

[0054] An analysis unit configured to perform big data analysis on the oilfield logging package data to obtain spliced data, data attributes, data types, and flow types;

[0055] A data compression unit configured to perform data compression on the spliced data and the data attributes to obtain attribute compression data;

[0056] A bit compression unit configured to perform bit compression on the attribute compression data and the data types to obtain type compression data;

[0057] A storage unit configured to obtain data storage location according to the type compression data and the flow types.

[0058] Advantages of the present application:

[0059] The present application provides a big data-based oilfield logging data management method, which decodes multiple oilfield logging package data formats and classifies big data to ensure data uniformity, performs data compression and bit compression to improve oilfield logging data compression performance and read-write efficiency, and ultimately improves logging data management efficiency.

[0060] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 is a big data-based oilfield logging data management method flowchart provided by the present application;

[0062] Figure 2 is a data splicing schematic diagram of the big data-based oilfield logging data management method provided by the present application;

[0063] Figure 3 is a big data analysis schematic diagram of the big data-based oilfield logging data management method provided by the present application;

[0064] Figure 4 is a bit encoding unit schematic diagram of the big data-based oilfield logging data management method provided by the present application;

[0065] Figure 5 is a module block diagram of an oilfield logging data management method based on big data provided by an embodiment of the application. DETAILED DESCRIPTION

[0066] The application will be further described in detail below in conjunction with specific embodiments, but the embodiments of the application are not limited thereto.

[0067] Embodiment one

[0068] In one specific embodiment, please refer to Figure 1 , Figure 1 is a flowchart of an oilfield logging data management method based on big data, and the specific steps are as follows:

[0069] S1: Oilfield logging data is the key information in oil and gas exploration and development, which contains the physical property data of underground rock layers and is crucial for identifying oil and gas reservoirs. With the progress of exploration technology, a large amount of logging data has been accumulated, and effective methods are needed to manage and utilize these data. In order to establish a complete data foundation for subsequent data analysis and processing, ensure the integrity and accuracy of the data, and provide reliable data support for oilfield exploration and development, oilfield logging encapsulated data is obtained.

[0070] S2: Due to the large amount and complexity of logging data, big data analysis technology is needed to integrate and analyze to extract valuable information. Through big data analysis, the integration of logging data is realized, and the efficiency and accuracy of data processing are improved. In one specific embodiment, please refer to Figure 2 , Figure 2 is a big data analysis diagram of an oilfield logging data management method based on big data, so that the oilfield logging encapsulated data is analyzed by big data to obtain splicing data, data attributes, data types, and flow types.

[0071] S21: Oilfield logging encapsulated data is usually stored in a specific format to ensure data integrity and security. In order to obtain raw data, a corresponding decoder is used according to the oilfield logging encapsulated data format to obtain raw data. In one specific embodiment, for LAS format, DLIS format, WIS format and other data formats, professional software tools such as LISview, Geolog, etc. are used to directly decode to obtain raw data.

[0072] S22: The raw data is analyzed by big data to obtain data attributes and flow types. In one specific embodiment, the specific steps are as follows:

[0073] The data attributes include various raw data attributes and various processed data attributes.

[0074] Through big data analysis technology, various original data attributes are obtained, such as flow data of underlying test data, acoustic travel time of conventional logging curves, bulk density of conventional logging curves, etc.

[0075] Through big data analysis technology, various processed data attributes are obtained from the original data, such as distribution characteristics, mean, and variance attributes of the data obtained through statistical analysis of the original data.

[0076] Through big data analysis technology, the read and write data volume of each network port is analyzed, and different data flow types are obtained, such as analyzing the read and write data types of different ports at different times according to the data volume to obtain the corresponding data flow type.

[0077] S23: Using big data classification on the data attributes to obtain data types, in one specific embodiment, the specific steps are as follows:

[0078] Using principal component analysis to extract the main features of the data attributes, reducing the dimensionality of the data, thereby facilitating classification;

[0079] Using clustering analysis method to classify according to the main features of the data attributes, through this method, data attributes with similar features can be classified into one category, thereby forming different data types;

[0080] In order to improve the accuracy and robustness of classification, ensure that the model can still make correct classification when facing new, unseen data, the relationship between data attributes and data types obtained by using neural network algorithm is used to optimize the mapping relationship between data attributes and data types.

[0081] S24: Obtaining spliced data according to the data attributes and the original data, in one specific embodiment, please refer to Figure 3 , Figure 3 It is a data splicing diagram of an oilfield logging data management method based on big data, the specific steps are as follows:

[0082] According to the provided data attributes, determine the dimensionality of the data, determine whether the data is single-point data, one-dimensional data, or two-dimensional data;

[0083] If it is single-point data, it means that each data point is independent and does not form a sequence or matrix with other data points, so the original data can be directly spliced according to the pre-set order or rule to form spliced data, this splicing is usually linear and does not need to consider the correlation between data points;

[0084] If it is one-dimensional data, such as time series or linearly arranged data points, the correlation of the original data needs to be further distinguished. If the data points have correlation, one-dimensional data can be processed by single-pointing first, that is, the linear structure of the data is maintained, and the spliced data is further formed by simple concatenation. On the contrary, if the data points have no correlation or need to be analyzed from different angles, one-dimensional data can be two-dimensionalized, that is, the time series data is converted into a two-dimensional representation of state-time by constructing a data matrix.

[0085] If it is two-dimensional data, the correlation of the original data needs to be distinguished. If the correlation of the original data indicates that the data should be maintained on a two-dimensional plane, the size of the data can be expanded by splicing rows or columns to form a larger two-dimensional data set. On the contrary, if information needs to be extracted from a more complex structure or to better represent the depth and hierarchy of the data, two-dimensional data can be three-dimensionalized, that is, a new dimension is added to represent different attributes or characteristics, thereby forming a three-dimensional data structure.

[0086] S3: Due to the huge amount of oilfield logging data, the direct storage and transmission cost is high, and the efficiency is low. Therefore, under the limited transmission bandwidth, the data needs to be compressed to improve the transmission efficiency. Since the compression technology can reduce the storage space and transmission bandwidth demand of the data, improve the economy and efficiency of data processing, maintain the integrity and usability of the data, and reduce the loss of the data in the transmission process, the data compression is performed on the spliced data and the data attributes to obtain attribute compressed data.

[0087] S31: According to the spliced data and the data attributes, a prediction operator, a transformation operator and an encoding operator are obtained.

[0088] S311: According to the spliced data, a data dimension is obtained. In one specific embodiment, the data dimension includes one-dimensional data, two-dimensional data and three-dimensional data.

[0089] S312: According to the data dimension, a prediction method, a transformation method and an encoding method are obtained. In one specific embodiment,

[0090] The prediction method is to remove the spatial correlation according to the data dimension and the spatial data correlation analysis, including a direction prediction method, a maximum value prediction method, a dictionary prediction method and a clustering prediction method.

[0091] The transformation method is to remove the frequency domain data correlation by using a frequency set method in combination with the prediction result and the statistical characteristics of the data, specifically including a discrete cosine transform, a discrete sine transform and a HAAR wavelet transform.

[0092] The encoding method is to combine the results of prediction and transformation and statistical characteristics of data, and to assign shorter code words to coefficients with higher frequency after transformation and longer code words to coefficients with lower frequency, so as to realize data compression, including exponential Golomb encoding, run-length encoding, Huffman encoding and direct transmission encoding.

[0093] S313: obtaining a prediction unit, a transformation unit and an encoding unit according to the data attribute and the data dimension, in a specific embodiment, the specific steps are:

[0094] Obtaining data space correlation according to the data attribute and the data dimension, potential correlation patterns in data can be found through statistical analysis and data mining techniques, for example, certain features may exhibit strong correlation in time or space, and this information is crucial for subsequent unit division;

[0095] The prediction unit is used to obtain the size of the prediction unit and the prediction subunit division according to the data space correlation;

[0096] The transformation unit is used to obtain the size of the transformation unit and the transformation subunit division according to the size of the prediction unit and the prediction subunit division;

[0097] The encoding unit is used to obtain the size of the encoding unit and the encoding subunit division according to the size of the transformation unit and the transformation subunit division in the data compression or encoding stage.

[0098] S314: obtaining a prediction operator according to the prediction method and the prediction unit, in a specific embodiment, the specific steps of the prediction operator are:

[0099] Obtaining a first-level prediction residual of the prediction unit by using different prediction methods for each prediction subunit;

[0100] Obtaining a prediction residual by using a prediction method for the first-level prediction residual of the prediction unit.

[0101] S315: obtaining a transformation operator according to the transformation method and the transformation unit, in a specific embodiment, the specific steps of the transformation operator are:

[0102] Obtaining a first-level transformation residual of the transformation unit by using a transformation method for each transformation subunit;

[0103] Obtaining a transformation residual by using a transformation method for the first-level transformation residual of the transformation unit.

[0104] S316: obtaining an encoding operator according to the encoding method and the encoding unit, in a specific embodiment, the transformation operator is to obtain a bit stream by using different encoding methods for each encoding subunit.

[0105] S32: obtaining a data compression method according to the splicing data and the prediction operator, the transformation operator and the encoding operator.

[0106] S321: obtaining a data analysis result by using data analysis on the splicing data, in a specific embodiment, the data analysis result includes the strength and range of data correlation.

[0107] S322: obtaining a prediction type, a transformation type and an encoding type according to the data analysis result, preferably, in a specific embodiment, the specific steps are as follows:

[0108] obtaining a plurality of alternative prediction methods according to the strength of data correlation, for example, for strongly correlated data, it is more suitable to use the extreme value prediction method, and for weakly correlated data, it is necessary to consider the direct transmission prediction method, obtaining a plurality of alternative prediction units according to the range of data correlation, and obtaining a plurality of prediction types according to the plurality of alternative prediction methods and the plurality of alternative prediction units;

[0109] obtaining a plurality of alternative transformation methods according to the strength of data correlation, obtaining a plurality of alternative transformation units according to the range of data correlation, and obtaining a plurality of transformation types according to the plurality of alternative transformation methods and the plurality of alternative transformation units;

[0110] obtaining a plurality of alternative encoding methods according to the strength of data correlation, obtaining a plurality of alternative encoding units according to the range of data correlation, and obtaining a plurality of encoding types according to the plurality of alternative encoding methods and the plurality of alternative encoding units.

[0111] S323: obtaining a prediction compression method according to the prediction type and the prediction operator, in a specific embodiment, the specific steps are as follows:

[0112] performing a preliminary selection in the plurality of prediction operators according to the plurality of prediction types to obtain a standby prediction compression method;

[0113] using the standby prediction compression method to perform a trial run on data to obtain a standby prediction compression method with the smallest prediction residual, and using the standby prediction compression method as the prediction compression method.

[0114] S324: obtaining a transformation compression method according to the transformation type and the transformation operator, in a specific embodiment, the specific steps are as follows:

[0115] performing a preliminary selection in the plurality of transformation operators according to the plurality of transformation types to obtain a standby transformation compression method;

[0116] The data is tested by using the backup transform compression method to obtain a backup transform compression method with the minimum transform residual, and the backup transform compression method is used as the transform compression method.

[0117] S325: obtaining an encoding compression method according to the encoding type and the encoding operator, in a specific embodiment, the specific steps are as follows:

[0118] According to the plurality of encoding types, a preliminary selection is performed in the plurality of encoding operators to obtain a backup encoding compression method;

[0119] The test data is compressed by using the backup encoding compression method to obtain a backup encoding compression method with the minimum code stream, and the backup encoding compression method is used as the encoding compression method.

[0120] S326: obtaining a data compression method according to the prediction compression method, the transform compression method and the encoding compression method, in a specific embodiment, the data compression method comprises the prediction compression method, the transform compression method and the encoding compression method connected in sequence.

[0121] S33: obtaining type compression data by using the data compression method on the spliced data.

[0122] S4: In order to further optimize the data storage and transmission efficiency, especially in the scene of limited bandwidth or limited storage space, bit compression can more finely control the compression degree of data, adapt to different data types and application requirements, realize efficient storage and fast access of data, and improve the overall performance of data processing, therefore, the attribute compression data and the data type are subjected to bit compression to obtain type compression data.

[0123] S41: obtaining an index encoding operator and a bit encoding operator according to the data type;

[0124] S411: obtaining a data attribute number and a data attribute feature according to the data type;

[0125] S412: obtaining an index encoding method and a bit encoding method according to the data attribute number and the data attribute feature, in a specific embodiment, the specific steps are as follows:

[0126] According to the data attribute number, an index encoding method of each attribute is obtained, including an equal bit encoding method and an exponential encoding method;

[0127] According to the data attribute number and the data attribute feature, a bit encoding method is obtained, including adaptive binary arithmetic coding and context adaptive variable length coding.

[0128] S413: obtaining a bit encoding unit according to the data type and the data attribute quantity, in one specific embodiment, please refer to Figure 4 , Figure 4 is a bit encoding unit schematic diagram of an oilfield logging data management method based on big data, and the specific steps are as follows:

[0129] obtaining multiple minimum bit encoding units according to multiple data attribute quantities;

[0130] obtaining a bit encoding unit according to the data type and multiple minimum bit encoding units by using the minimum multiple principle and feature analysis.

[0131] S414: obtaining an index encoding operator according to the index encoding method, in one specific embodiment, each index encoding method corresponds to an index encoding operator.

[0132] S415: obtaining a bit encoding operator according to the bit encoding method and the bit encoding unit, in one specific embodiment, the specific steps of the bit encoding operator are as follows:

[0133] obtaining a first-level code stream by using a bit encoding method for each bit encoding unit;

[0134] obtaining a storage code stream by using a bit encoding method for the first-level code stream.

[0135] S42: obtaining a bit compression method according to the attribute compression data, the index encoding operator and the bit encoding operator;

[0136] S421: obtaining a compression analysis result by using compression analysis for the attribute compression data, in one specific embodiment, the compression analysis result includes a compression data size and a bit correlation strength;

[0137] S422: obtaining an index encoding type and a bit encoding type according to the compression analysis result, in one specific embodiment, the specific steps are as follows:

[0138] obtaining a corresponding index encoding type according to the compression data size;

[0139] obtaining multiple alternative bit encoding methods according to the bit correlation strength, for example, for continuous data with strong correlation, it is more suitable to use run-length encoding; and for discrete data with weak correlation, Huffman encoding can be more suitable, in this way, the most suitable bit encoding type can be selected for different data characteristics and application scenarios, so as to realize efficient data compression and transmission;

[0140] obtaining multiple bit encoding types according to multiple alternative bit encoding methods and bit encoding units.

[0141] S423: obtaining a bit compression method according to the index encoding type, the index encoding operator, the bit encoding type and the bit encoding operator, in a specific embodiment, the specific steps are as follows:

[0142] According to the index encoding type, selecting from a plurality of index encoding operators, the selection process usually involves artificial decision-making, which may be based on experience, previous research results or specific application requirements, to obtain an index encoding compression method;

[0143] According to a plurality of bit encoding types, a plurality of bit encoding operators are selected to obtain a backup bit encoding compression method;

[0144] Trial run the data using the backup bit encoding compression method to actually generate the smallest code stream of one of the backup bit encoding compression methods;

[0145] Combining the index encoding compression method and the backup bit encoding compression method with the smallest code stream in the trial run, the final bit compression method can be obtained, which combines the advantages of index encoding and bit encoding, aiming to realize efficient compression of data, index encoding provides fast data access and retrieval capability, while bit encoding is responsible for minimizing the storage space of data. Through this integrated method, data can be accessed while achieving high compression ratio, meeting the efficiency requirements of storage and transmission.

[0146] S43: obtaining type compression data by using the bit compression method on the attribute compression data.

[0147] S5: Because the storage location of data directly affects the access speed and processing efficiency of data, it is necessary to optimize the storage location according to the characteristics and access mode of data, and different storage strategies are needed for different types of data to achieve the best performance and cost-effectiveness. Considering that the optimal data storage location can be determined according to the traffic type of data and the characteristics of compressed data, so as to not only improve the access speed and processing efficiency of data, but also ensure the safe storage of data, finally reduce the storage cost and improve the flexibility of data management, therefore, the data storage location is obtained according to the type compression data and the traffic type.

[0148] S51: obtaining a primary index according to the traffic type, in a specific embodiment, the traffic type is indexed in ascending order according to the traffic type of data transmission, such as IP traffic and port traffic.

[0149] S52: obtaining a secondary index according to the type compression data, in a specific embodiment, the type compression data is indexed in ascending order according to the compression and decompression complexity and energy consumption of the compressed data.

[0150] S53: obtaining a data storage location according to the primary index and the secondary index, in a specific embodiment, according to the physical layout of the storage system, the organization mode of the data and the access mode, the data corresponding to the primary index and the secondary index is stored in a distributed manner, and at the same time, as the data access mode changes, the primary index and the secondary index need to be dynamically updated to optimize the storage and access efficiency of the data.

[0151] The oilfield logging data management method based on big data of the embodiment can ensure the uniformity of data through decoding of various oilfield logging packaging data formats and big data classification; through data compression and bit compression, the compression performance and read-write efficiency of the oilfield logging data are improved, and finally the efficiency of logging data management is improved.

[0152] In a specific embodiment, please refer to Figure 5 , Figure 5 A module block diagram of an oilfield logging data management method based on big data, an oilfield logging data management system based on big data, comprising:

[0153] The acquisition unit is used for acquiring oilfield logging packaging data, and the unit includes a modern network communication and data storage system.

[0154] The analysis unit is used for obtaining spliced data, data attributes, data types and flow types by using big data analysis on the oilfield logging packaging data, and the unit integrates high-performance FPGA and SOC chips to realize hardware acceleration of data analysis and effectively process big data flow of the oilfield logging packaging data.

[0155] The data compression unit is used for obtaining attribute compression data by using data compression on the spliced data and the data attributes, and the hardware architecture of the unit includes a compression module and a decompression module, which can quickly process parallel data compression of the write port and parallel data decompression of the read port.

[0156] The bit compression unit is used for obtaining type compression data by using bit compression on the attribute compression data and the data types, and the hardware architecture of the unit includes a compression module and a decompression module, which can process data processing of the data center in parallel.

[0157] The storage unit is used for obtaining a data storage location according to the type compression data and the flow type.

[0158] The terms "first", "second", etc. are used only for the purpose of description and do not connote or imply any relative importance or imply a specific number of features. Thus, features defined with "first", "second" can include one or more of the features explicitly or implicitly. In the description of the present application, the meaning of "a plurality" is two or more, unless otherwise expressly specified.

[0159] Although the present application has been described herein in relation to particular embodiments thereof, many other variations and modifications and other uses will become apparent to those skilled in the art upon a review of the description of the present application, the drawings and the appended claims. In the claims, the term comprising does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality.

[0160] The above description is further to make further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those skilled in the art of the present application, a number of simple deductions or replacements can be made without departing from the concept of the present application, which should be regarded as falling within the protection scope of the present application.

Claims

1. A big data-based oilfield logging data management method, characterized in that, The method comprises the following steps: obtaining oilfield logging package data; performing big data analysis on the oilfield logging package data to obtain spliced data, data attributes, data types and flow types; performing data compression on the spliced data and the data attributes to obtain attribute compressed data; performing bit compression on the attribute compressed data and the data types to obtain type compressed data; obtaining a data storage location according to the type compressed data and the flow types; the step of performing big data analysis on the oilfield logging package data to obtain spliced data, data attributes, data types and flow types comprises the following steps: obtaining original data according to the oilfield logging package data format using a corresponding decoder; obtaining data attributes and flow types by performing big data analysis on the original data; obtaining data types by performing big data classification on the data attributes; splicing the original data according to the data attributes to obtain spliced data; the step of performing data compression on the spliced data and the data attributes to obtain attribute compressed data comprises the following steps: obtaining a prediction operator, a transformation operator and an encoding operator according to the spliced data and the data attributes; obtaining a data compression method according to the spliced data, the prediction operator, the transformation operator and the encoding operator; obtaining type compressed data by performing data compression on the spliced data; the step of obtaining a data compression method according to the spliced data and the prediction operator, the transformation operator and the encoding operator comprises the following steps: obtaining a data analysis result by performing data analysis on the spliced data; obtaining a prediction type, a transformation type and an encoding type according to the data analysis result; obtaining a prediction compression method according to the prediction type and the prediction operator; obtaining a transformation compression method according to the transformation type and the transformation operator; obtaining an encoding compression method according to the encoding type and the encoding operator; obtaining a data compression method according to the prediction compression method, the transformation compression method and the encoding compression method; the step of performing bit compression on the attribute compressed data and the data types to obtain type compressed data comprises the following steps: obtaining an index encoding operator and a bit encoding operator according to the data types; obtaining a bit compression method according to the attribute compressed data, the index encoding operator and the bit encoding operator; obtaining type compressed data by performing the bit compression method on the attribute compressed data; the step of obtaining a bit compression method according to the attribute compressed data, the index encoding operator and the bit encoding operator comprises the following steps: obtaining a compression analysis result by performing compression analysis on the attribute compressed data; obtaining an index encoding type and a bit encoding type according to the compression analysis result; obtaining a bit compression method according to the index encoding type, the index encoding operator, the bit encoding type and the bit encoding operator.

2. The big data-based oilfield logging data management method according to claim 1, characterized in that, the step of obtaining a prediction operator, a transformation operator and an encoding operator according to the spliced data and the data attributes comprises the following steps: obtaining a data dimension according to the spliced data; obtaining a corresponding prediction method, a transformation method or an encoding method according to the data dimension; obtaining a prediction unit, a transformation unit and an encoding unit according to the data attributes and the data dimension; According to the prediction method and the prediction unit, a prediction operator is obtained; According to the transformation method and the transformation unit, a transformation operator is obtained; According to the encoding method and the encoding unit, an encoding operator is obtained.

3. The method of claim 1, wherein, The index encoding operator and the bit encoding operator according to the data type comprises: According to the data type, a data attribute number and a data attribute feature are obtained; According to the data attribute number and the data attribute feature, an index encoding method and a bit encoding method are obtained; According to the data type and the data attribute number, a bit encoding unit is obtained; According to the index encoding method, an index encoding operator is obtained; According to the bit encoding method and the bit encoding unit, a bit encoding operator is obtained.

4. The big data-based oilfield logging data management method according to claim 1, characterized in that, The data storage location according to the type compression data and the flow type comprises: According to the flow type, a first index is obtained; According to the type compression data, a second index is obtained; According to the first index and the second index, a data storage location is obtained.

5. A big data based oilfield logging data management system characterized in that, The method according to any one of claims 1-4, comprising: a collection unit configured to acquire oilfield logging package data; an analysis unit configured to perform big data analysis on the oilfield logging package data to obtain spliced data, data attributes, data types, and flow types; a data compression unit configured to perform data compression on the spliced data and the data attributes to obtain attribute compression data; a bit compression unit configured to perform bit compression on the attribute compression data and the data types to obtain type compression data; a storage unit configured to obtain a data storage location according to the type compression data and the flow type.

Images