Oil and gas field data anomaly detection method, device and equipment based on multiple models and medium
Through multi-model methods, the data scenarios of oil and gas fields and intelligent abnormality detection are solved, and the problem of artificial experience dependence in the existing technology is achieved, and efficient and intelligent data quality management is achieved.
Patent Information
- Application Number
- CN202510482883.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-17
- Publication Date
- 2025-08-01
AI Technical Summary
In the prior art, oil and gas field data abnormality detection mainly relies on manual experience, and is relatively low in intelligence, making it difficult to meet the needs of real-time, high-dimensional diversity and dynamic adaptability.
A multi-model-based method is adopted to obtain oil and gas field data and preprocess, identify the scene type, match the corresponding clustering model or regression analysis model, and combine the adaptive feedback mechanism to perform abnormal detection, including data acquisition, preprocessing, scene recognition, model training and adaptive feedback.
It realizes intelligent abnormal detection of oil and gas field data, improves the accuracy and intelligence of detection, reduces dependence on manual experience, adapts to changes in the data environment, and improves the real-time and efficiency of data quality management.
Smart Images
Figure CN120408433A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of oil and gas field data processing, and in particular, to a method, device, equipment and medium for abnormal detection of oil and gas field data based on multiple models. Background Art
[0002] With the acceleration of digital transformation, data has become the core driving force for production and management activities.
[0003] In the process of oil and gas exploration, development and production, high-quality data is crucial for optimizing production processes, reducing risks and enhancing competitiveness. Abnormal data detection is essential for safe production and efficient development.
[0004] However, related technologies mainly rely on manual experience to detect abnormal oil and gas field data, with a low degree of intelligence. Summary of the Invention
[0005] The present invention provides a method, device, equipment and medium for abnormal detection of oil and gas field data based on multiple models, which is used to solve the defect that related technologies mainly rely on manual experience to detect abnormal oil and gas field data, with a low degree of intelligence, and can realize intelligent abnormal detection of oil and gas field data, and improve the intelligence level of abnormal detection of oil and gas field data.
[0006] In a first aspect, the present invention provides a method for abnormal detection of oil and gas field data based on multiple models, including: Obtaining the data to be detected obtained by data collection and preprocessing in an oil and gas field, and obtaining the use of the data to be detected; Determining the scene type corresponding to the data to be detected according to the data content of the data to be detected and the use of the data to be detected; Based on the scene type corresponding to the data to be detected, determining a matching target processing model among multiple trained data processing models; Using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result; Detecting whether the data to be detected is abnormal according to the processing result and the set threshold data.
[0007] Optionally, the scene type corresponding to the data to be detected includes a spatial scene and / or a depth scene; and the multiple data processing models include a clustering model and a regression analysis model.
[0008] Optionally, the determining the scene type corresponding to the data to be detected according to the data content of the data to be detected and the use of the data to be detected includes: If the data content of the data to be detected is the first data content including geographical coordinates and operation unit identifiers, and the data to be detected is used to analyze the characteristic differences among multiple operation units in the oil and gas field, then determine that the scenario type corresponding to the data to be detected is a spatial scenario; the operation unit is an operation area or a well site; If the data content of the data to be detected is the second data content, and it is determined that the data to be detected is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected is a depth scenario; wherein, the second data content is that the parameter values of physical parameters change with depth; If the data content of the data to be detected includes the first data content and the second data content, the first data content is used to analyze the characteristic differences among multiple operation units in the oil and gas field, and the second data content is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario.
[0009] Optionally, determining a matching target processing model from multiple trained data processing models based on the scenario type corresponding to the data to be detected includes: Obtain pre-created associated data, where the associated data includes a one-to-one correspondence between multiple scenario types and multiple data processing models; Based on the scenario type corresponding to the data to be detected, search in the associated data for the data processing model corresponding to the scenario type, and determine the found data processing model as the target processing model; Wherein, when the scenario type corresponding to the data to be detected is a spatial scenario, the target processing model is a clustering model; when the scenario type corresponding to the data to be detected is a depth scenario, the target processing model is a regression analysis model; when the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario, the target processing model includes a clustering model and a regression model.
[0010] Optionally, when the data content of the data to be detected is the first data content and the target processing model is a clustering model, the threshold data is a distance threshold; Using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result includes: Using the clustering model to cluster the data to be detected to obtain multiple clustering centers; Detecting whether the data to be detected is abnormal according to the processing result and the set threshold data includes: Calculate the distance between each clustering center and a preset normal clustering center; If the distance is greater than the distance threshold, determine that the data to be detected is abnormal data; If the distance is not greater than the distance threshold, determine that the data to be detected is normal data.
[0011] Optionally, when the data content of the data to be detected is the second data content and the target processing model is a regression analysis model, the threshold data is a deviation threshold; The using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result includes: Using the regression analysis model to perform regression analysis on the data to be detected to obtain a corresponding predicted physical parameter value; The detecting whether the data to be detected is abnormal according to the processing result and the set threshold data includes: Calculating the deviation between the predicted physical parameter value and the actual measured value; If the deviation is greater than the deviation threshold, determine that the data to be detected is abnormal data; If the deviation is not greater than the deviation threshold, determine that the data to be detected is normal data.
[0012] Optionally, when the data content of the data to be detected includes the first data content and the second data content and the target processing model includes a clustering model and a regression analysis model, the threshold data includes an error threshold; The using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result includes: Using the clustering model to cluster the first data content in the data to be detected to obtain a plurality of cluster centers; and, using the regression analysis model to perform regression analysis on the second data content in the data to be detected to obtain a corresponding predicted physical parameter value; The detecting whether the data to be detected is abnormal according to the processing result and the set threshold data includes: Determining the distance between each cluster center and the normal cluster center, and determining the deviation between the predicted physical parameter value and the actual measured value; Performing weighted summation on the distance and the deviation based on the set weight to obtain a corresponding total error; If the total error is greater than the error threshold, determine that the data to be detected is abnormal data; If the total error is not greater than the error threshold, determine that the data to be detected is normal data.
[0013] In a second aspect, the present invention provides an oil and gas field data anomaly detection device based on multiple models, including: An acquisition unit, configured to acquire the data to be detected obtained by data collection and preprocessing in an oil and gas field, and acquire the purpose of the data to be detected; A scenario determination unit, configured to determine the scenario type corresponding to the data to be detected according to the data content of the data to be detected and the purpose of the data to be detected; A model determination unit, configured to determine a matching target processing model from multiple trained data processing models based on the scenario type corresponding to the data to be detected; A data processing unit, configured to perform corresponding processing on the data to be detected using the target processing model to obtain a processing result; An anomaly detection unit, configured to detect whether the data to be detected is abnormal according to the processing result and the set threshold data.
[0014] In a third aspect, the present invention provides a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to execute the multi-model-based oil and gas field data anomaly detection method according to the first aspect or any corresponding implementation manner thereof.
[0015] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the multi-model-based oil and gas field data anomaly detection method according to the first aspect or any corresponding implementation manner thereof.
[0016] The multi-model-based oil and gas field data anomaly detection method, device, equipment and medium provided by the present invention acquire the data to be detected obtained by data collection and preprocessing in an oil and gas field, and acquire the purpose of the data to be detected. According to the data content of the data to be detected and the purpose of the data to be detected, determine the scenario type corresponding to the data to be detected. Based on the scenario type corresponding to the data to be detected, determine a matching target processing model from multiple trained data processing models. Use the target processing model to perform corresponding processing on the data to be detected to obtain a processing result. According to the processing result and the set threshold data, detect whether the data to be detected is abnormal. The present invention can achieve intelligent anomaly detection of oil and gas field data and improve the intelligent level of oil and gas field data anomaly detection. Description of the Drawings
[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0018] Figure 1 Flow chart of a multi - model - based oil and gas field data anomaly detection method provided by an embodiment of the present invention; Figure 2 Flow chart of another multi - model - based oil and gas field data anomaly detection method provided by an embodiment of the present invention; Figure 3 Structure schematic diagram of a multi - model - based oil and gas field data anomaly detection device provided by an embodiment of the present invention; Figure 4 Structure schematic diagram of a computer device provided by an embodiment of the present invention. Detailed implementation manners
[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.
[0020] The following combines Figure 1 - Figure 2 Describe the multi - model - based oil and gas field data anomaly detection method of the present invention.
[0021] As Figure 1 shown, the first multi - model - based oil and gas field data anomaly detection method is proposed in this embodiment, and this method may include the following steps: S101. Obtain the data to be detected obtained by data collection and pre - processing in the oil and gas field, and obtain the use of the data to be detected.
[0022] Specifically, in the process of oil and gas exploration in this embodiment, multi - category data in the oil and gas field, namely oil and gas field data such as well location, well deviation angle, porosity and lithology, are collected by downhole logging instruments and ground data collection systems, and are uploaded to the data center or cloud in real - time or in batches to ensure the integrity and timeliness of the data.
[0023] Specifically, this embodiment can pre - process the collected oil and gas field data to obtain the pre - processed data as the data to be detected. Among them, data pre - processing may include duplicate removal and merging, missing value filling, normalization, standardization and feature engineering, etc.
[0024] Among them, in the deduplication and merging of this embodiment, duplicate records and redundant information collected multiple times can be identified, and a deduplication algorithm is used to merge duplicate data points. In the filling of missing values, linear interpolation and spline interpolation are used to complete local missing values, and collaborative filtering algorithms or industry experience models are used to fill large-area missing values. In the normalization and standardization, standard score or minimum-maximum normalization methods can be used to ensure the unity of data dimensions. In the feature engineering, geographical coordinate conversion can be performed, converting longitude and latitude into plane coordinates, calculating statistical values such as well depth and well deviation difference features and cumulative thickness, and extracting industry-specific features such as multi-well correlation and lithology difference.
[0025] S102. Determine the scenario type corresponding to the data to be detected according to the data content of the data to be detected and the use of the data to be detected.
[0026] Specifically, this embodiment can identify the scenario type corresponding to the data to be detected based on the data content of the data to be detected and the use of the data to be detected.
[0027] Optionally, the scenario type corresponding to the data to be detected includes a spatial scenario and / or a depth scenario; among the multiple data processing models include a clustering model and a regression analysis model.
[0028] Among them, the clustering model can be a K-means clustering or hierarchical clustering model, and the regression analysis model can be a random forest regression model, an Extreme Gradient Boosting (XGBoost) model, or a Support Vector Regression (SVR) model.
[0029] Optionally, step S102 may include: If the data content of the data to be detected is the first data content including geographical coordinates and operation unit identifiers, and the data to be detected is used to analyze the characteristic differences between multiple operation units in the oil and gas field, then determine that the scenario type corresponding to the data to be detected is a spatial scenario; the operation unit is an operation area or a well site; If the data content of the data to be detected is the second data content, and it is determined that the data to be detected is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected is a depth scenario; where the second data content is that the parameter values of physical parameters change with depth; If the data content of the data to be detected includes the first data content and the second data content, the first data content is used to analyze the characteristic differences between multiple operation units in the oil and gas field, and the second data content is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario.
[0030] Specifically, when the data to be detected contains geographical coordinates, regional identifiers (such as block numbers) or wellsite identifiers (such as well numbers), and aims to analyze cross-regional performance comparisons (such as production differences) or inter-well characteristic differences (such as production capacity fluctuations), the scenario type corresponding to the data to be detected is determined as a spatial scenario. If the data to be detected changes continuously with well depth / stratigraphy and it is necessary to study depth-related characteristics (such as pressure gradient, lithology distribution), then the scenario type corresponding to the data to be detected is determined as a depth scenario.
[0031] Specifically, the scenario recognition rules of this embodiment include: spatial scenario, where the data to be detected contains geographical coordinates or longitude and latitude, mainly focusing on well group distribution and regional anomalies; depth scenario, where the data changes continuously with well depth or stratigraphic horizons, and it is necessary to capture the variation law of the data with depth; composite scenario, which contains both geographical and depth information, and uses multi-algorithm parallel processing or multi-stage analysis.
[0032] S103. Based on the scenario type corresponding to the data to be detected, determine the matching target processing model among multiple trained data processing models.
[0033] Specifically, in this embodiment, data collection and preprocessing can be performed in an oil and gas field in advance to obtain sample data for training the data processing model. The pre-trained data processing model is trained using the sample data and the corresponding true measurement values to optimize the model performance of the pre-trained data processing model until its performance meets the requirements, and a trained data processing model is obtained.
[0034] Specifically, during the model training process of this embodiment, training data preparation can be carried out first. Priority is given to selecting expert-confirmed or historical high-quality logging data as the training set, and regression training is performed by establishing normal labels, abnormal labels, or fitting true values (such as the average porosity measured manually). Then, model evaluation and parameter tuning can be carried out. The silhouette coefficient, Calinski-Harabasz index, or Davies-Bouldin index can be used to measure the compactness and separation of clustering in clustering evaluation; in regression evaluation, indicators such as root mean square error, root mean square deviation, coefficient of determination, and mean absolute error are used to measure the fitting degree and prediction accuracy. In addition, the best hyperparameters, such as the number of trees and maximum depth of a random forest, can be automatically searched through grid search, Bayesian optimization, or the automated machine learning AutoML framework to improve the model performance. Then, this embodiment can verify and further optimize the model. After optimization, model deployment and version management can be carried out. The trained data processing model is packaged and deployed to an online environment (such as a microservice or container), and at the same time, a model version management mechanism is established to facilitate comparison of the detection accuracy of new and old models and support rollback or upgrade.
[0035] Optionally, step S103 may include: Obtain pre-created associated data, where the associated data includes the one-to-one correspondence between multiple scenario types and multiple data processing models; Based on the scenario type corresponding to the data to be detected, search for the data processing model corresponding to the scenario type in the associated data, and determine the found data processing model as the target processing model; Among them, when the scenario type corresponding to the data to be detected is a spatial scenario, the target processing model is a clustering model; when the scenario type corresponding to the data to be detected is a depth scenario, the target processing model is a regression analysis model; when the scenario type corresponding to the data to be detected includes both a spatial scenario and a depth scenario, the target processing model includes a clustering model and a regression model.
[0036] S104. Use the target processing model to perform corresponding processing on the data to be detected to obtain a processing result.
[0037] Specifically, after determining the target processing model in this embodiment, the data to be detected can be input into the target processing model for corresponding processing to obtain the processing result output by the target processing model.
[0038] S105. Detect whether the data to be detected is abnormal according to the processing result and the set threshold data.
[0039] Among them, the threshold data can be set by technicians according to the actual situation, and this embodiment does not make any limitations.
[0040] Optionally, in other multi-model-based oil and gas field data anomaly detection methods proposed in this embodiment, when the data content of the data to be detected is the first data content and the target processing model is a clustering model, the threshold data is a distance threshold. At this time, step S104 may include: Use the clustering model to cluster the data to be detected to obtain multiple cluster centers.
[0041] At this time, step S105 may include: Calculate the distance between each cluster center and the preset normal cluster center; If the distance is greater than the distance threshold, determine that the data to be detected is abnormal data; If the distance is not greater than the distance threshold, determine that the data to be detected is normal data.
[0042] Optionally, when the data content of the data to be detected is the second data content and the target processing model is a regression analysis model, the threshold data is a deviation threshold. At this time, step S104 may include: Use the regression analysis model to perform regression analysis on the data to be detected to obtain the corresponding predicted physical parameter value.
[0043] At this time, step S105 may include: Calculate the deviation between the predicted physical parameter value and the actual measured value; If the deviation is greater than the deviation threshold, determine the data to be detected as abnormal data; If the deviation is not greater than the deviation threshold, determine the data to be detected as normal data.
[0044] Optionally, when the data content of the data to be detected includes the first data content and the second data content, and the target processing model includes a clustering model and a regression analysis model, the threshold data includes an error threshold. At this time, step S104 may include: Use the clustering model to cluster the first data content in the data to be detected to obtain multiple cluster centers; and use the regression analysis model to perform regression analysis on the second data content in the data to be detected to obtain the corresponding predicted physical parameter value.
[0045] At this time, step S105 may include: Determine the distance between each cluster center and the normal cluster center, and determine the deviation between the predicted physical parameter value and the actual measured value; Based on the set weight, perform weighted summation on the distance and the deviation to obtain the corresponding total error; If the total error is greater than the error threshold, determine the data to be detected as abnormal data; If the total error is not greater than the error threshold, determine the data to be detected as normal data.
[0046] Specifically, when the scenario type corresponding to the data to be detected is a spatial scenario, this embodiment may adopt K-means clustering (or combined with density clustering, hierarchical clustering models, etc.) to identify the well group distribution characteristics and locate abnormal well points, which is particularly suitable for unlabeled scenarios and can adapt to different numbers of clusters.
[0047] Specifically, when the scenario type corresponding to the data to be detected is a depth scenario, this embodiment may use random forest regression (complementary to XGBoost, SVR, etc.), which is good at capturing non-linear and multi-dimensional feature interactions and has a certain robustness to noise and missing data.
[0048] Specifically, if the scenario type corresponding to the data to be detected includes both a spatial scenario and a depth scenario, this embodiment may improve the analysis effect through multi-model integrated detection. At this time, after predicting the data to be detected using different regression models and clustering models (such as linear regression, random forest, neural network), the weighted average method (assign weights according to the root mean square error and coefficient of determination values of model cross-validation) or the voting mechanism (weighted by confidence or majority vote rule in classification tasks) may be used to synthesize the results, and the advantages of the models are fused to characterize complex non-linear relationships, such as in scenarios of oil and gas production prediction or reservoir parameter inversion.
[0049] The method for detecting abnormal oil and gas field data based on multiple models proposed in this embodiment obtains the data to be detected obtained by data collection and preprocessing in the oil and gas field, as well as the purpose of the data to be detected. According to the data content of the data to be detected and the purpose of the data to be detected, the corresponding scenario type of the data to be detected is determined. Based on the scenario type corresponding to the data to be detected, a matching target processing model is determined among multiple trained data processing models. The target processing model is used to perform corresponding processing on the data to be detected to obtain a processing result. According to the processing result and the set threshold data, it is detected whether the data to be detected is abnormal. This embodiment can realize the intelligent abnormal detection of oil and gas field data and improve the intelligent level of oil and gas field data abnormal detection.
[0050] In the related art, with the acceleration of the digital transformation of various industries, data is gradually becoming the core driving force of production and management activities. In fields such as oil exploration, geological mapping, smart finance, and industrial Internet, real-time and high-quality data is crucial for optimizing production processes, reducing risks, and enhancing competitiveness. In fields such as oil and gas exploration and geological surveys, it is often necessary to collect various types of logging data (such as well azimuth, well inclination, porosity, lithology, etc.). Due to reasons such as equipment failures, transmission delays, multi-source heterogeneity, and manual entry errors, data often has problems such as missing values, redundancy, outliers, and inconsistent coding. The larger the data scale and the more complex the data structure, the more obvious the negative impact of these problems on analysis models and decision support systems. The periodic and rule-driven analysis methods in the related art often cannot meet the requirements of real-time monitoring and dynamic optimization. Currently, there are the following problems with the data: First, the data sources are inconsistent: the diversification of equipment and instruments leads to differences in format, accuracy, and storage methods; field conflicts or missing values are likely to occur when integrating data from multiple platforms. Second, the data quality is uneven: missing values and outliers frequently appear, and the regularity is not strong; sensor failures or environmental noise interference result in unstable data distributions. Third, the scene dependence is strong: the requirements for algorithms in the spatial dimension (such as wellhead geographical coordinates) and the depth dimension (such as the change of well inclination and porosity with well depth) are not the same; the rule determination methods in the related art cannot fully capture the coupling relationship between multi-dimensional features.
[0051] The data quality detection methods in the related art mainly include detection methods based on fixed rules and thresholds, statistical analysis methods, and machine learning methods, but each has its limitations. The method based on fixed rules and thresholds relies on expert experience to set detection rules and judges anomalies by exceeding the preset range, but it has insufficient recognition of complex non-linear or hidden anomaly phenomena, and the cost of rule adjustment is relatively high. The statistical analysis method uses indicators such as mean, variance, and correlation coefficient to detect anomalies, which is effective for linear features or obvious deviations, but has limited detection ability in high-dimensional, non-linear, or strongly coupled data environments. The machine learning methods in the related art use algorithms such as decision trees and support vector machines for classification or regression prediction, but have insufficient modeling capabilities for time series relationships, dynamic environments, and feature engineering, and it is difficult to adapt to complex scenarios. These methods are difficult to balance real-time performance, high-dimensional diversity, and dynamic adaptability, and it is difficult to meet the rapidly changing business requirements in the big data environment. Therefore, the related art needs a system solution that can automatically learn, continuously optimize, and integrate multiple algorithm models to improve detection accuracy, reduce dependence on manual experience, and achieve all-round management and monitoring of data quality.
[0052] In this embodiment, by automatically matching applicable algorithms, combining online / offline training for high-accuracy anomaly detection, and adopting an adaptive feedback mechanism to optimize the model, the intelligent improvement of data quality management can be effectively realized.
[0053] As Figure 2 shown, this embodiment can first prepare sample data for training the model. Specifically, this embodiment can perform data collection and preprocessing, such as collecting downhole and surface data, uploading it to the data center for preprocessing such as deduplication and missing value handling, and performing normalization, standardization, and feature engineering to obtain sample data. After that, this embodiment can perform model training and verification based on the sample data to obtain a trained data processing model. Anomaly detection and quality assessment are performed on the data to be detected based on the data processing model.
[0054] Among them, when the scenario type corresponding to the sample data in this embodiment is a spatial scenario, and after using the clustering model to cluster the data to be detected to obtain the clustering center, the distance between it and the normal clustering center or density peak can be calculated. If it exceeds the set threshold, it is marked as an outlier anomaly, and the outlier score is output. At the same time, a visualization chart (such as a map heat map) is generated to intuitively display the anomaly distribution, and an anomaly heat map is generated to show the geographical distribution of the anomaly well points.
[0055] In this embodiment, when the scene type corresponding to the data to be detected is a depth scene, and the physical parameter prediction value has been obtained by performing regression analysis on the data to be detected using a regression analysis model, it can be compared with the actual measurement value. If the deviation exceeds the error upper limit, it is determined as abnormal. At the same time, hierarchical early warning is carried out in combination with the error distribution and displayed in a visual manner, such as the curve comparison with depth or well inclination, and the abnormal situation changing with depth is intuitively displayed through the curve comparison chart.
[0056] This embodiment can also generate a quality assessment report. Specifically, this embodiment can comprehensively evaluate the data health status, including overall indicators such as the abnormal rate, average deviation, and abnormal position distribution, and provide key analysis and solution suggestions for severely abnormal or high-risk areas to optimize data quality management.
[0057] This embodiment can also set up an adaptive feedback mechanism. For the detected abnormal situations, this embodiment can confirm the problem source in combination with the operation and maintenance records or the results of manual review, and correct the rules and models through the feedback mechanism, including dynamically updating the rule thresholds, retraining the model, or optimizing the hyperparameters. Through this closed-loop system, the detection accuracy of the system is continuously improved in continuous iteration to adapt to the new data environment, and the model accuracy is continuously improved through the dynamic update of the rule thresholds and the iterative optimization of the hyperparameters.
[0058] This embodiment can also perform model incremental update and retraining. This embodiment can trigger incremental training when the new data accumulates to a certain threshold or the model detection effect drops to the warning value. If the data distribution drifts significantly, full-scale retraining is performed and the model version is updated. Through continuous iterative training, the adaptive ability of the model to changes in the production environment is improved.
[0059] This embodiment can also perform automated deployment. This embodiment can combine with the continuous integration / continuous delivery process to realize the automated online and monitoring of the model, and ensure the stability and reliability of the system by monitoring operation metrics (such as latency, throughput, error rate, etc.).
[0060] This embodiment can also perform offline and online integration. Among them, the offline part is responsible for large-scale data preprocessing, batch model training, and rule management, while the online part receives data streams in real time, calls the trained model for prediction and monitoring, and issues timely warnings and provides processing solutions when abnormalities occur. Through a unified data management platform, the complementarity and collaboration between offline and online are realized, and the overall efficiency of the system is improved.
[0061] This embodiment can also set up a visualization interface and a reporting system. Specifically, this embodiment can provide an intuitive front-end visualization interface to display real-time anomaly detection results, alarm levels, and the prediction situations of each model, and regularly generate a risk assessment report covering anomaly distribution, changes in important features, and rule adjustments brought about by system self-learning. At the same time, it supports multi-terminal access and message push, facilitating decision-makers and business personnel to quickly obtain data quality information and take timely actions.
[0062] This embodiment can automatically match algorithm models such as clustering and regression through scenario recognition, and can make full use of the advantages of machine learning in spatial distribution and depth distribution scenarios. Through joint prediction of multiple models, it can comprehensively capture the non-linear and multi-dimensional correlations of data, thus significantly improving the accuracy of anomaly detection. When the data distribution changes over time or with the environment, the system can automatically adapt and maintain a high detection accuracy. And it can be extended to more algorithms to handle diverse data patterns.
[0063] From data collection to anomaly alarm, the entire process of this embodiment can be implemented automatically or semi-automatically, greatly reducing labor costs; adopting a modular architecture, it can be quickly migrated and deployed in more industries and large data environments. An automatic learning closed-loop is established, and the model is continuously iterated through incremental updates or full-scale retraining.
[0064] In addition to oil and gas exploration, this embodiment can also be applied to multi-dimensional complex data scenarios such as mineral resource exploration, engineering geological monitoring, and smart city geographic information systems; it also has the adaptability to scenarios with high requirements for real-time or near-real-time (such as online monitoring systems).
[0065] This embodiment can solve the deficiencies in the data quality detection methods in the related technologies in terms of real-time performance, data utilization rate, and adaptive ability. Aiming at problems such as inconsistent data sources, uneven quality, and strong scenario dependence, this embodiment integrates a variety of advanced algorithms to achieve efficient data quality management. This embodiment can perform data collection and preprocessing, scenario recognition and algorithm selection, model training and verification, anomaly detection and quality assessment, adaptive feedback and model update, system integration and application, etc., and has a complete set of functions and operation processes. This embodiment can use machine learning technology to improve the accuracy of anomaly detection, achieve full-process automation or semi-automation, reduce labor costs, support rapid migration and deployment, and achieve continuous optimization through an automatic learning closed-loop, and is applicable to a variety of data quality management scenarios.
[0066] The multi-model-based abnormal detection method for oil and gas field data proposed in this embodiment can achieve multi-model integrated detection, comprehensively predict oil and gas field data using a variety of data processing models, and synthesize the final result by weighted or voting methods, improving the ability to depict complex non-linear relationships. It realizes automatic data preprocessing and feature extraction, completes data cleaning, missing value filling, and feature engineering through automatic or semi-automatic methods, greatly reducing human intervention and improving data utilization efficiency. A real-time abnormal detection and feedback optimization mechanism is set up, which not only realizes real-time monitoring of data streams, but also can dynamically update rule thresholds and models according to abnormal feedback, forming an adaptive closed-loop system. It realizes system integration and application, deeply couples offline model training and online real-time monitoring, improving the overall practicality and scalability of the system; facing a variety of industry scenarios, it has high generality, strong portability, and significant economic and social benefits.
[0067] As Figure 3 shown, this embodiment proposes a multi-model-based abnormal detection device for oil and gas field data, which may include: An acquisition unit 301, configured to acquire the data to be detected obtained by data collection and preprocessing in an oil and gas field, and acquire the purpose of the data to be detected; A scenario determination unit 302, configured to determine the scenario type corresponding to the data to be detected according to the data content of the data to be detected and the purpose of the data to be detected; A model determination unit 303, configured to determine a matching target processing model from multiple trained data processing models based on the scenario type corresponding to the data to be detected; A data processing unit 304, configured to perform corresponding processing on the data to be detected using the target processing model to obtain a processing result; An abnormal detection unit 305, configured to detect whether the data to be detected is abnormal according to the processing result and the set threshold data.
[0068] It should be noted that the processing processes and the beneficial effects brought by the acquisition unit 301, the scenario determination unit 302, the model determination unit 303, the data processing unit 304, and the abnormal detection unit 305 can respectively refer to Figure 1 the steps S101 to S105 in, and will not be elaborated here.
[0069] Optionally, the scenario type corresponding to the data to be detected includes a spatial scenario and / or a depth scenario; the multiple data processing models include a clustering model and a regression analysis model.
[0070] Optionally, the scenario determination unit 302 is further configured to: If the data content of the data to be detected is the first data content including geographical coordinates and operation unit identifiers, and the data to be detected is used to analyze the characteristic differences between multiple operation units in the oil and gas field, then determine that the scenario type corresponding to the data to be detected is a spatial scenario; the operation unit is an operation area or a well site; If the data content of the data to be detected is the second data content, and it is determined that the data to be detected is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected is a depth scenario; wherein, the second data content is that the parameter values of physical parameters change with depth; If the data content of the data to be detected includes the first data content and the second data content, the first data content is used to analyze the characteristic differences between multiple operation units in the oil and gas field, and the second data content is used to determine the depth correlation feature, then determine that the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario.
[0071] Optionally, the model determination unit 303 is further configured to: Obtain pre-created association data, where the association data includes a one-to-one correspondence between multiple scenario types and multiple data processing models; Based on the scenario type corresponding to the data to be detected, search in the association data for the data processing model corresponding to the scenario type, and determine the found data processing model as the target processing model; Wherein, when the scenario type corresponding to the data to be detected is a spatial scenario, the target processing model is a clustering model; when the scenario type corresponding to the data to be detected is a depth scenario, the target processing model is a regression analysis model; when the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario, the target processing model includes a clustering model and a regression model.
[0072] Optionally, when the data content of the data to be detected is the first data content and the target processing model is a clustering model, the threshold data is a distance threshold; The data processing unit 304 is further configured to: Use the clustering model to cluster the data to be detected to obtain multiple cluster centers; The anomaly detection unit 305 is further configured to: Calculate the distance between each cluster center and a preset normal cluster center; If the distance is greater than the distance threshold, determine that the data to be detected is abnormal data; If the distance is not greater than the distance threshold, determine that the data to be detected is normal data.
[0073] Optionally, when the data content of the data to be detected is the second data content and the target processing model is a regression analysis model, the threshold data is a deviation threshold; The data processing unit 304 is further configured to: Use a regression analysis model to perform regression analysis on the data to be detected, and obtain the corresponding predicted physical parameter values; The anomaly detection unit 305 is further configured to: Calculate the deviation between the predicted physical parameter value and the actual measured value; If the deviation is greater than the deviation threshold, determine that the data to be detected is abnormal data; If the deviation is not greater than the deviation threshold, determine that the data to be detected is normal data.
[0074] Optionally, when the data content of the data to be detected includes a first data content and a second data content, and the target processing model includes a clustering model and a regression analysis model, the threshold data includes an error threshold; The data processing unit 304 is further configured to: Use the clustering model to cluster the first data content in the data to be detected to obtain multiple cluster centers; and use the regression analysis model to perform regression analysis on the second data content in the data to be detected to obtain the corresponding predicted physical parameter values; The anomaly detection unit 305 is further configured to: Determine the distance between each cluster center and the normal cluster center, and determine the deviation between the predicted physical parameter value and the actual measured value; Based on the set weight, perform weighted summation on the distance and the deviation to obtain the corresponding total error; If the total error is greater than the error threshold, determine that the data to be detected is abnormal data; If the total error is not greater than the error threshold, determine that the data to be detected is normal data.
[0075] In a second aspect, the present invention provides an oil and gas field data anomaly detection device based on multiple models, including: An acquisition unit 301, configured to acquire the data to be detected obtained by performing data acquisition and preprocessing in an oil and gas field, and acquire the use of the data to be detected; A scenario determination unit 302, configured to determine the scenario type corresponding to the data to be detected according to the data content of the data to be detected and the use of the data to be detected; A model determination unit 303, configured to determine a matching target processing model from multiple trained data processing models based on the scenario type corresponding to the data to be detected; A data processing unit 304, configured to perform corresponding processing on the data to be detected using the target processing model to obtain a processing result; An anomaly detection unit 305, configured to detect whether the data to be detected is abnormal according to the processing result and the set threshold data.
[0076] The multi-model-based oil and gas field data anomaly detection device proposed in this embodiment obtains the data to be detected obtained by data collection and preprocessing in the oil and gas field, as well as the purpose of obtaining the data to be detected. According to the data content of the data to be detected and the purpose of the data to be detected, the corresponding scenario type of the data to be detected is determined. Based on the scenario type corresponding to the data to be detected, a matching target processing model is determined among multiple trained data processing models. The target processing model is used to perform corresponding processing on the data to be detected to obtain a processing result. According to the processing result and the set threshold data, it is detected whether the data to be detected is abnormal. This embodiment can achieve intelligent anomaly detection of oil and gas field data and improve the intelligent level of oil and gas field data anomaly detection.
[0077] The multi-model-based oil and gas field data anomaly detection device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.
[0078] An embodiment of the present invention also provides a computer device having the above Figure 3 multi-model-based oil and gas field data anomaly detection device shown.
[0079] Please refer to Figure 4 , a schematic structural diagram of a computer device provided by an optional embodiment of the present invention. The computer device includes: one or more processors 10, a memory 20, and an interface for connecting each component, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 4 One processor 10 is taken as an example in
[0080] The processor 10 can be a central processor, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above hardware chip can be an application specific integrated circuit, a programmable logic device, or a combination thereof. The above programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.
[0081] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.
[0082] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function. The data storage area can store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely provided with respect to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.
[0083] ; The memory 20 may include a volatile memory, such as a random access memory. The memory may also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive. The memory 20 may also include a combination of the above types of memories.
[0084] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
[0085] The embodiments of the present invention further provide a computer-readable storage medium. The method according to the embodiments of the present invention can be implemented in hardware, firmware, or can be implemented as computer code recorded on a storage medium, or can be implemented as computer code originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and to be stored in a local storage medium, so that the method described herein can be processed by such software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.
[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An oil and gas field data anomaly detection method based on multiple models, characterized in that, Including: Obtaining the data to be detected obtained by data collection and preprocessing in the oil and gas field, and obtaining the purpose of the data to be detected; Determining the corresponding scenario type of the data to be detected according to the data content of the data to be detected and the purpose of the data to be detected; Based on the scenario type corresponding to the data to be detected, determining a matching target processing model among multiple trained data processing models; Using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result; Detecting whether the data to be detected is abnormal according to the processing result and the set threshold data.
2. The method according to claim 1, characterized in that The scenario type corresponding to the data to be detected includes a spatial scenario and / or a depth scenario; the multiple data processing models include a clustering model and a regression analysis model.
3. The method according to claim 2, characterized in that, The determining the corresponding scenario type of the data to be detected according to the data content of the data to be detected and the purpose of the data to be detected includes: If the data content of the data to be detected is the first data content including geographical coordinates and an operation unit identifier, and the data to be detected is used to analyze the characteristic differences between multiple operation units in the oil and gas field, then determining that the corresponding scenario type of the data to be detected is a spatial scenario; the operation unit is an operation area or a well site; If the data content of the data to be detected is the second data content, and it is determined that the data to be detected is used to determine depth correlation features, then determining that the corresponding scenario type of the data to be detected is a depth scenario; wherein, the second data content is that the parameter value of the physical parameter changes with depth; If the data content of the data to be detected includes the first data content and the second data content, the first data content is used to analyze the characteristic differences between multiple operation units in the oil and gas field, and the second data content is used to determine depth correlation features, then determining that the corresponding scenario type of the data to be detected includes a spatial scenario and a depth scenario.
4. The method according to claim 3, characterized in that, The determining a matching target processing model among multiple trained data processing models based on the scenario type corresponding to the data to be detected includes: Obtaining pre-created association data, where the association data includes a one-to-one correspondence between multiple scenario types and multiple data processing models; Based on the scenario type corresponding to the data to be detected, searching for the data processing model corresponding to the scenario type in the association data, and determining the found data processing model as the target processing model; Wherein, when the scenario type corresponding to the data to be detected is a spatial scenario, the target processing model is a clustering model; when the scenario type corresponding to the data to be detected is a depth scenario, the target processing model is a regression analysis model; when the scenario type corresponding to the data to be detected includes a spatial scenario and a depth scenario, the target processing model includes a clustering model and a regression model.
5. The method according to claim 4, wherein When the data content of the data to be detected is the first data content and the target processing model is a clustering model, the threshold data is a distance threshold; The using the target processing model to perform corresponding processing on the data to be detected to obtain a processing result includes: Cluster the to-be-detected data using the clustering model to obtain multiple cluster centers; Detecting whether the to-be-detected data is abnormal according to the processing result and the set threshold data includes: Calculate the distance between each of the cluster centers and a preset normal cluster center; If the distance is greater than the distance threshold, determine that the to-be-detected data is abnormal data; If the distance is not greater than the distance threshold, determine that the to-be-detected data is normal data.
6. The method according to claim 4, wherein When the data content of the to-be-detected data is the second data content and the target processing model is a regression analysis model, the threshold data is a deviation threshold; Using the target processing model to perform corresponding processing on the to-be-detected data to obtain a processing result includes: Perform regression analysis on the to-be-detected data using the regression analysis model to obtain corresponding predicted physical parameter values; Detecting whether the to-be-detected data is abnormal according to the processing result and the set threshold data includes: Calculate the deviation between the predicted physical parameter value and the actual measured value; If the deviation is greater than the deviation threshold, determine that the to-be-detected data is abnormal data; If the deviation is not greater than the deviation threshold, determine that the to-be-detected data is normal data.
7. The method according to claim 4, characterized in that When the data content of the to-be-detected data includes the first data content and the second data content and the target processing model includes a clustering model and a regression analysis model, the threshold data includes an error threshold; Using the target processing model to perform corresponding processing on the to-be-detected data to obtain a processing result includes: Cluster the first data content in the to-be-detected data using the clustering model to obtain multiple cluster centers; and perform regression analysis on the second data content in the to-be-detected data using the regression analysis model to obtain corresponding predicted physical parameter values; Detecting whether the to-be-detected data is abnormal according to the processing result and the set threshold data includes: Determine the distance between each of the cluster centers and the normal cluster center, and determine the deviation between the predicted physical parameter value and the actual measured value; Perform weighted summation on the distance and the deviation based on the set weight to obtain a corresponding total error; If the total error is greater than the error threshold, determine that the to-be-detected data is abnormal data; If the total error is not greater than the error threshold, determine that the to-be-detected data is normal data.
8. An oil and gas field data anomaly detection device based on multiple models, characterized in that, Includes: An acquisition unit for acquiring the to-be-detected data obtained by performing data acquisition and preprocessing in an oil and gas field, and acquiring the use of the to-be-detected data; A scenario determination unit for determining the scenario type corresponding to the to-be-detected data according to the data content of the to-be-detected data and the use of the to-be-detected data; An anomaly detection unit for detecting whether the data to be detected is anomalous according to the processing result and set threshold data.
9. A computer device, characterized in that, Comprising: A memory and a processor, which are communicatively connected to each other. Computer instructions are stored in the memory, and the processor executes the computer instructions to execute the multi-model based oil and gas field data anomaly detection method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the multi-model based oil and gas field data anomaly detection method according to any one of claims 1 to 7.