A method, equipment, and storage medium for fusing and processing real-time data from multi-component heterogeneous oil and gas production.

Abstract

A method, device, and storage medium for the fusion processing of real-time data from diverse and heterogeneous oil and gas production are disclosed, comprising the following steps: S1: Unified collection of real-time production data from multiple independent systems; data is then input into a database and stored via a device driver layer; S2: Unified integration of data from various systems, standardization of data encoding and storage, and standardization of data push to external systems; S3: A unified data service interface is provided externally by the platform; S4: Business function integration. This enables the access, preprocessing, and monitoring of diverse and heterogeneous data, deploys a unified data warehouse, and achieves hierarchical classification, centralized storage, and unified management of data.

Description

Technical Field

[0001] This invention belongs to the field of oil and gas field surface gathering and transportation technology, specifically relating to a method, equipment, and storage medium for the fusion processing of real-time data of multi-element heterogeneous oil and gas production. Background Technology

[0002] Oil and gas field surface gathering and transportation stations are built by integrating information technology, the Internet of Things, big data, and other technologies with natural gas development, production, and management. However, to date, they suffer from a large number and variety of self-built systems, such as video monitoring systems, intelligent bubble drainage systems, intelligent well switching systems, and leak monitoring systems. These systems are used extensively but operate independently, resulting in multiple data collection points, redundant data collection, and a significant amount of heterogeneous data from various sources. This leads to long command response times and low efficiency, hindering the rapid advancement of digital transformation in oil and gas surface gathering and transportation management.

[0003] Based on the requirements of digital transformation and the existing problems, it is essential to integrate the existing independent systems such as video surveillance, site inspection, bubble drainage and injection, well switching and leak monitoring with the existing surface gathering and transportation information systems. This will allow for the fusion and interconnection of data from each system, the deepening of application functions, centralized remote control of related businesses, and optimization of the integrated collaborative management model for oil and gas field surface gathering and transportation operations, thereby achieving the goal of digital transformation in surface gathering and transportation management. Summary of the Invention

[0004] To address the problems existing in the prior art, the objective of this invention can be achieved through the following technical solution: An industrial big data platform is constructed using multiple protocols, integrating data from various independent systems into the platform. This enables data collection and integration from automated and information systems, consolidating real-time production data, IoT data, equipment monitoring and operation data, and gas volume prediction data. This facilitates the access, preprocessing, and monitoring of diverse and heterogeneous data. A unified data warehouse is deployed, enabling hierarchical classification, centralized storage, and unified management of data. On the upper-level platform, an object-oriented model is used to reorganize and apply data from multiple sources, enabling unified analysis and processing. This gradually forms a data-centric approach that guides business operations and promotes improved operational efficiency and effectiveness within enterprises / departments.

[0005] A method for fusing and processing real-time data from multi-component heterogeneous oil and gas production specifically includes the following steps:

[0006] S1: Unified collection of real-time production data from multiple independent systems; data is then entered into a database and stored through the device driver layer.

[0007] S2: Unify and integrate data from various systems, standardize data encoding and storage, and standardize data push to external systems.

[0008] S3: The platform provides a unified data service interface to the outside world.

[0009] S4: Business Function Integration

[0010] It enables the access, preprocessing, and monitoring of diverse and heterogeneous data, deploys a unified data warehouse, and achieves hierarchical classification, centralized storage, and unified management of data.

[0011] Further improvements, step S1: specifically include the following steps:

[0012] S101: Determine the data collection scope and method. By deploying platform collectors, establish a protocol driver layer to uniformly interface with various industrial devices, IoT devices, video devices, and other devices using different communication protocols.

[0013] S1011: The independent system includes a production data SCADA system, a production video monitoring system, an intelligent bubble drainage system, an intelligent well switching system, and a leak detection system (laser pan-tilt unit).

[0014] S1012: The data collection range for each independent system is as follows:

[0015] Production video surveillance system: basic data, business data

[0016] Intelligent bubble bath system: status information, real-time data

[0017] Intelligent well control system: Equipment information

[0018] Leakage Detection System (Laser Pan-Tilt-Zone): Equipment Information

[0019] S1013: The communication protocol of the independent system corresponding to the protocol driver layer established by the platform collector is as follows:

[0020] Production data SCADA system modbus-TCP

[0021] Production video surveillance system TCP / IP

[0022] Intelligent bubble extraction system Modbus-TCP

[0023] Modbus-RTU Intelligent Switching Well System

[0024] Leakage monitoring system (laser PTZ) modbus-TCPS102: Data acquisition function design and acquisition driver management are carried out according to the data acquisition range and acquisition method of each independent system. It supports common acquisition protocols, including RESTful API, Webservice, Modbus, MQTT, RS485, CPCUA, TCP / IP and other protocols.

[0025] Data Acquisition Point Management: Configuration protocol management, batch import of acquisition points, data acquisition point maintenance and management. Data Simulator: Simulated data point creation, batch creation of simulated points.

[0026] Video capture management: Configure video source information such as name, alias, video type, address type, description, and URL. Source data display: Display source data values ​​in a simple table format for easy real-time display during access, facilitating quick confirmation of the correctness of point source and point table configuration information.

[0027] System configuration management: Configure data acquisition service information, set port number, UUID, etc., set redundancy and breakpoint resume, device cache storage, etc.

[0028] System operation log: View user login and usage; view the operation status of collectors, drivers, and point sources; query by time and content;

[0029] Step 2 involves data governance, data standard management, and data fusion processing of the collected data from each independent system. The specific steps are as follows:

[0030] S201: Data Cleaning

[0031] Data cleaning addresses issues such as inconsistent, duplicate, missing, and outdated data, as well as uncorrelated metadata, by performing unified cleaning.

[0032] For real-time data, erroneous data is labeled and alerts are issued, and erroneous data is mapped to an "error database" for subsequent manual screening and processing.

[0033] For structured data, perform screening and alerts for erroneous data, duplicate data, and dead zone data, and process them according to rules.

[0034] S202: Analysis and Calculation

[0035] After cleaning, the data is processed again according to the calculation rules / model. The platform can calculate the corresponding data using batch calculation, statistical calculation, real-time calculation, and streaming calculation methods.

[0036] S203: Data Quality Management

[0037] Record the detection results and alarm results of abnormal data, manage the error information of the data in a targeted manner, identify and analyze the error information according to the code of the error data, and eliminate or modify the error information.

[0038] S204: Data Security Data Desensitization Management: Sensitive information is transformed through desensitization rules to achieve reliable protection of sensitive privacy data.

[0039] Data encryption management: Setting encryption rules for critical information.

[0040] Data approval process management: Data modifications require relevant permission approvals. The approval process for data modifications can be configured according to business needs.

[0041] S205: Data Quality Management

[0042] In step 3, the platform provides the following unified data service interfaces to external users:

[0043] Production Data SCADA System: OPC

[0044] Production Video Surveillance System API

[0045] Intelligent Bubble Extraction System OPC

[0046] Intelligent Switching Well System OPC

[0047] Leakage Detection System (Laser Pan / Tilt) OPC

[0048] Step 4 also includes business function integration, specifically including the following steps:

[0049] S401: Build a unified portal login engine on the production network side: Implement enterprise portal functions through various application interfaces, single sign-on, unified authentication and other methods to provide users with a unified entry point for personalized access to various information and service resources.

[0050] S402: The functional interfaces of independent systems such as the production video monitoring system, intelligent bubble drainage system, intelligent well switching system, and leakage monitoring system (laser PTZ) are transformed into microservices, with a unified data warehouse providing data services to form a unified application platform on the production network side. This includes the following steps:

[0051] S4021: Determine the integration method and scope of business functions. This will result in functions such as data query, display, monitoring and alarm, remote control, data analysis and decision-making information display, multi-dimensional topic analysis, and custom topic data mining.

[0052] S4022: Building a Technical Framework for Business Function Integration

[0053] (5) Service framework: Spring Cloud Alibaba was selected as the scaffolding for the microservice architecture solution;

[0054] (6) Core Support Components

[0055] Service Registry: Supports service discovery, dynamic configuration, service health monitoring, and service governance. A unified health check dashboard allows for the management of service availability and traffic based on health status.

[0056] API Gateway: Supports dynamic routing rules and traffic control; supports peak request flow control and allows defining various rate limiting rules; supports recording request and response data, API latency analysis, and performance monitoring; supports authentication, data masking, traffic cleaning, backend signing, API high availability, and flexible fault tolerance mechanisms.

[0057] Configuration Center: Dynamic configuration services manage application and service configurations across all environments in a centralized, externalized, and dynamic manner. Dynamic configuration eliminates the need to redeploy applications and services when configurations change, making configuration management more efficient and agile. Centralized configuration management simplifies the implementation of stateless services and makes on-demand elastic scaling of services easier.

[0058] (7) Service Security

[0059] A role-based authentication and authorization mechanism introduces digital certificates to sign and verify communication data, ensuring user identity authentication and the integrity of communication information. Digital signatures and timestamps guarantee the non-repudiation of business operations.

[0060] (8) Other support services

[0061] Console: Provides developers with system management and parameter configuration functions, and provides system administrators and operations personnel with log level adjustment, cache update and viewing functions, task scheduling management functions, and abnormal task query functions.

[0062] Monitoring Center: Collects data sent by applications, performs statistics and analysis, and provides it to operations and maintenance personnel in the form of graphical pages.

[0063] Service management platform: Provides functions for service registration, discovery, subscription, service on / off, service rate limiting, service degradation, and service weight adjustment.

[0064] Business Domain Open Platform: Provides developers with functions such as project creation, resource download, interface testing, and package upload.

[0065] S403: Microservice applications, including

[0066] (1) General-purpose components

[0067] Applications that need to be integrated are first horizontally split according to the generality of the components to form general service components for the platform.

[0068] (2) Business service components

[0069] First, the application is vertically split into modules based on business domains. These modules can be integrated and deployed within a single application, or deployed separately as individual modules or combinations of several modules. Then, each module is horizontally split, following a layered design based on interface programming standards, such as a business interface layer, business implementation layer, data access interface layer, data access implementation layer, and remote service invocation layer.

[0070] (3) Process Services

[0071] Define process services to implement business functions. When a process service depends on two or more atomic services, service orchestration is performed through a service orchestration framework. In the process service layer, a compensated TCC distributed transaction model is used to implement distributed transactions at the service layer. Process services register services and publish services externally, supporting at least HTTP / JSON. Process services can call each other.

[0072] (4) Service Interface

[0073] Throughout the architecture design process, an interface-oriented programming approach is adopted to implement the open / closed principle (open for extension, closed for modification) at the system architecture design level. When new functions are added to the system, it is not necessary to modify the existing system structure and code.

[0074] Using the method of this invention, the following was achieved:

[0075] (1) Data Integration

[0076] Data integration includes enabling data sharing and exchange between various application systems, and mainly includes:

[0077] 1) Unified Data Interface. Establish a unified data interface management system to enable unified registration, maintenance, and querying of data integration interfaces for existing systems, and construct a unified view of data integration interfaces.

[0078] 2) Review the existing data integration status and determine the appropriate interface specifications based on different business needs, so that the data integration construction of future application systems will have a basis to follow.

[0079] 3) For future business systems under construction, in principle, no new integration platforms will be added to form a unified data exchange and a unified integration interface.

[0080] (2) Application Integration

[0081] Based on a microservice architecture that conforms to mainstream industry standards, various business application systems are reorganized and optimized, functions with the same purpose are managed and merged in a unified manner, and application systems with multiple functions are split as needed.

[0082] (3) Enterprise information portal integration

[0083] Unified portal login engine. Various application interfaces, single sign-on, unified authentication, and other methods are used to implement enterprise portal functions, providing users with a unified and personalized entry point to access various information and service resources.

[0084] (4) Data storage, analysis and computing integration

[0085] By leveraging data warehousing and data analysis and visualization technologies such as online analytical processing (OLAP), data mining, and front-end visualization, we provide business analysis and decision support functions for management and executive levels, thereby enhancing enterprise management capabilities.

[0086] 1) From a technical perspective, this includes the planning, design, and implementation of two main aspects: data analysis engine and visualization analysis front-end.

[0087] 2) From a functional perspective, this includes functions such as KPI analysis and decision-making information display, multi-dimensional thematic analysis, data mining, and operational monitoring.

[0088] 3) From the perspective of analytical methods. A four-pronged analytical approach is adopted, including: digital analysis, graphical analysis, textual analysis, and process analysis. The combination of these four approaches provides decision-makers with accurate and clear decision-making basis.

[0089] Compared with existing solutions, the solution of the present invention can achieve the following expected effects:

[0090] (1) Centralized management and control of business operations improves on-site work efficiency.

[0091] Integrating multiple application systems into a single management platform enables modular management of these systems, avoiding the fragmented and isolated deployment and management of each system. This reduces the difficulty of managing different systems, while allowing the native application functions of each system to be switched and displayed freely on the industrial data platform. This provides convenience for operators and managers, thereby enabling lightweight management and application of each system.

[0092] (2) Optimize data management and enhance the value of data utilization.

[0093] The industrial data management platform extracts massive amounts of data from the underlying application system into a database. Through data governance, modeling, analysis, and application, the data optimizes internal management and improves business, while externally releasing the value of data cooperation. At the same time, data aggregation and integration better support predictive analysis, cross-domain analysis, proactive analysis, real-time analysis, and diversified structured data analysis, accelerating the process from data to value and building corresponding business capabilities.

[0094] (3) Break down data barriers and reduce data usage costs

[0095] By integrating data from different systems, all application systems share a single database with the industrial data platform, truly achieving data sharing between systems and breaking down "information silos" and "data barriers." This reduces the implementation costs of application systems while efficiently utilizing data resources. The industrial data platform's excellent scalability ensures that subsequent application systems can be easily and quickly integrated into the platform, reducing the resources invested in system installation and deployment.

[0096] The present invention also includes a computer device comprising a processor and a memory, wherein the memory stores a computer program, and the processor executes the computer program to implement the method of the present invention.

[0097] The present invention also includes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention. Attached Figure Description

[0098] Figure 1 This is a general framework diagram of the present invention.

[0099] Figure 2 This is a technical architecture diagram of the present invention.

[0100] Figure 3 This is a schematic diagram of the integrated application of laser methane alarm according to the present invention.

[0101] Figure 4 Laser-guided methane leak location analysis view Detailed Implementation

[0102] In the following, the terms “comprising” or “may include” as used in various embodiments of the invention indicate the presence of an inventive function, operation, or element, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of the invention, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or adding one or more combinations of the foregoing.

[0103] The terminology used in the various embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of the invention. Unless otherwise specified, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. The terms (such as those defined in commonly used dictionaries) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning unless clearly defined in the various embodiments of the invention.

[0104] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0105] This method utilizes mature and advanced IT technologies to build unified production control and office collaboration platforms. Within the oil and gas field production network, it integrates and consolidates various real-time data from production, video, and early warning monitoring, providing a unified foundational environment for data analysis and intelligent applications for field-related operations such as production scheduling, on-site monitoring and control, and safety management. Within the oil and gas field office network, based on the group company's Dream Cloud platform technical architecture, it integrates the company's unified systems, enabling employees to access multiple systems with a single sign-on. Self-built systems, developed using a component-based, modular, and microservice-based approach, form an "application store." Users in different positions can subscribe to their own job portals from the application store, deeply satisfying the personalized needs of different roles.

[0106] Overall architecture as follows Figure 1 As shown. A platform for fusing real-time data from multi-source, heterogeneous oil and gas production is established, specifically including:

[0107] Production network work portal and personalized work portal;

[0108] Among them, the production network work portal includes the production network environment, and its business application layer consists of an integrated intelligent control center, including: business integration functions such as electronic inspection, video supervision, remote control, emergency response, production adjustment, dynamic analysis, remote collaboration, leakage early warning, and automatic filling; as well as IT-enabled trend early warning, big data analysis, risk identification, proactive removal, artificial intelligence, intelligent diagnosis, virtual metering, digital twin, and engineering early warning applications;

[0109] Its basic environment layer includes: an industrial data platform, which stores and extracts data through an established industrial real-time database, enabling trend analysis, data analysis modeling, data acquisition, industrial reports, and smart industrial APP functions; its data sources come from the Internet of Things, RTU, OPC / modbs, video data, and SCADAS / FG.

[0110] A personalized work portal includes the office network environment;

[0111] Its business application layer consists of an integrated collaborative work platform, including: a self-built supplementary system consisting of a single map integration, intelligent workflow, document management, safety and environmental protection, emergency command, integrity management, and comprehensive office, as well as a unified business system consisting of a work area management platform, a development zone management platform, pipeline management, production management, production command platform, collaborative work platform, planning and scheduling, and ERP.

[0112] Its basic environment layer includes: the Dream Cloud Platform, which consists of a container platform, microservice framework, software development pipeline, middleware, and upstream business platform, and a regional data lake consisting of data collection, including data standardization, data quality management, metadata, master data, data model / information collection and monitoring, and on-site dynamic data;

[0113] Its data source comes from production data collected by monitoring equipment obtained from the production data platform.

[0114] 1 Data Fusion

[0115] Data fusion involves the unified collection, storage, and service of data from multiple independent systems. This includes determining the scope and methods of data collection, designing data collection functions, and ensuring data governance and security. It breaks down data silos in the production network by collecting and storing data at the device driver layer. The data from each system is then uniformly integrated, standardized in terms of data encoding and storage, and standardized in terms of data pushed externally. The platform then provides a unified data service interface to the outside world.

[0116] The specific steps are as follows:

[0117] 1. Determine the data collection scope and method. By deploying platform collectors, establish a protocol driver layer to uniformly interface with various industrial devices, IoT devices, video devices, etc., using different communication protocols. A single collector can collect 50,000 data points, and each collector can establish three collection drivers: an OPC DA driver for 30,000 points or an OPC UA driver for 10,000 data points.

[0118] This method currently requires data collection from 5 independent systems. The data collection methods are shown in the table below.

[0119] Table 1. Data Collection Methods for Production Network

[0120]

[0121]

[0122] The data collection range for each independent system is shown in the table below:

[0123] Table 2 Data Collection Scope

[0124]

[0125]

[0126] 2. Design the data acquisition function based on the data acquisition scope and method of each independent system. See the table below:

[0127] Table 2 Data Acquisition and Management Function Table

[0128]

[0129]

[0130] Data governance is performed on the collected data from various independent systems. This ensures that the integrated and fused multi-source heterogeneous data possesses high quality, unified standards, and formats, meeting the needs of production management applications. Data governance is based on an industrial data warehouse, involving relevant data standard management and fusion processing, data cleaning, analysis and calculation, and data quality management. The data warehouse also provides unified data storage, management, and application services.

[0131] (1) Data standard management and integration processing

[0132] All types of data collected in a unified manner are stored in a data warehouse according to unified data standards. Through the formulation and release of unified data standards, combined with institutional constraints and system control, the integrity, effectiveness, consistency, standardization, openness and sharing of data on the integrated collaborative work platform are achieved, thereby improving the level of enterprise-level data governance.

[0133] (2) Data cleaning

[0134] Data cleaning addresses issues such as inconsistent, duplicate, missing, and outdated data, as well as uncorrelated metadata, by performing unified cleaning.

[0135] For real-time data, erroneous data is labeled and alerts are issued, and erroneous data is mapped to an "error database" for subsequent manual screening and processing.

[0136] For structured data, perform screening and alerts for erroneous data, duplicate data, and dead zone data, and process them according to rules.

[0137] (3) Analysis and Calculation

[0138] The cleaned data undergoes secondary processing according to preset calculation rules / models. For example, to obtain missing data, secondary calculations are performed on the data, processing it according to established formulas or models. The platform can calculate the corresponding data using batch calculation, statistical calculation, real-time calculation, and streaming calculation methods.

[0139] (4) Data quality management

[0140] Data quality management refers to a series of activities, including the planning, implementation, and control, that use relevant technologies to measure, improve, and ensure data quality. Data quality management primarily involves recording the detection results and alarm results of abnormal data, managing data error information in a targeted manner, identifying and analyzing error information based on the coding of the error data, and eliminating or modifying the error information.

[0141] 4. Data security

[0142] (1) Data anonymization management

[0143] Sensitive information is transformed using de-identification rules to reliably protect sensitive privacy data.

[0144] Table 3 Examples of Data Desensitization

[0145] data Zhang San (Sales Position) Li Si (Production Position) Wang Wu (Human Resources Position) pressure value *** 2.36 MPa *** Traffic value <![CDATA[5000m 3 ]]> <![CDATA[5000m 3 ]]> *** Employee ID **** **** 0057

[0146] (2) Data Encryption Management

[0147] Set encryption rules for critical information.

[0148] (3) Data audit process management

[0149] Data modification requires relevant permission approval. The approval process for data modification can be configured according to business needs.

[0150] 2. Integration of Business Functions

[0151] like Figure 2 The integration of business functions begins with building a unified portal login engine on the production network side. Enterprise portal functionality is implemented through various application interfaces, single sign-on, and unified authentication, providing users with a unified entry point for personalized access to various information and service resources.

[0152] Secondly, the functional interfaces of multiple systems, such as the production video monitoring system, intelligent bubble drainage system, intelligent switch well system, and leakage monitoring system (laser PTZ), are transformed into microservices, and data services are provided by a unified data warehouse to form a unified application platform on the production network side.

[0153] The specific processing steps are as follows:

[0154] 1. Determine the integration method and scope of business functions. Existing functions will be categorized, integrated, and transformed according to the needs of on-site production control, forming functions such as data query, display, monitoring and alarm, and remote control. Functions such as data analysis and decision-making information display based on a unified data warehouse, multi-dimensional thematic analysis, and custom thematic data mining will also be added. Specific functions are shown in the table below:

[0155] Table 4 List of Business Function Widgets (Micro-applications)

[0156]

[0157]

[0158] 2. Business Function Integration Technology Framework Design. Based on the microservice architecture specifications and considering factors such as the target business volume of the industrial data platform, existing technology platforms, and development language constraints (Java), the platform layer designs the following technical solution:

[0159] 1) Service Framework

[0160] A service framework is a one-stop solution for microservice development, and also a collection of microservice components. Spring Cloud Alibaba is chosen as the scaffolding for the microservice architecture solution. It includes the necessary components for developing a microservice architecture, making it easy for developers to use these components to develop microservice architectures through the microservice programming model. With just a few annotations and minimal configuration, microservice applications can be integrated into the industrial data platform.

[0161] 2) Core Support Components

[0162] Service Registry: Supports service discovery, dynamic configuration, service health monitoring, and service governance. A unified health check dashboard allows for the management of service availability and traffic based on health status.

[0163] API Gateway: Supports dynamic routing rules and traffic control; supports peak request flow control and allows defining various rate limiting rules; supports recording request and response data, API latency analysis, and performance monitoring; supports authentication, data masking, traffic cleaning, backend signing, API high availability, and flexible fault tolerance mechanisms.

[0164] Configuration Center: Dynamic configuration services manage application and service configurations across all environments in a centralized, externalized, and dynamic manner. Dynamic configuration eliminates the need to redeploy applications and services when configurations change, making configuration management more efficient and agile. Centralized configuration management simplifies the implementation of stateless services and makes on-demand elastic scaling of services easier.

[0165] Load balancing: Employs multiple load balancing components, including server-side hardware load balancing, server-side software load balancing, and client-side load balancing, to distribute requests from the business layer.

[0166] 3) Service security

[0167] The role-based authentication and authorization mechanism introduces digital certificates to sign and verify communication data, ensuring user identity authentication and the integrity of communication information. Digital signatures and digital timestamps ensure the non-repudiation of business operations.

[0168] 4) Other support services

[0169] To support rapid development, testing, deployment, release, operation management, and maintenance monitoring, it provides an open platform including a console, monitoring center, service registry, service management platform, and business domain-related components.

[0170] Console: Provides developers with system management and parameter configuration functions, and provides system administrators and operations personnel with log level adjustment, cache update and viewing functions, task scheduling management functions, and abnormal task query functions.

[0171] Monitoring Center: Collects data sent by applications, performs statistics and analysis, and provides it to operations and maintenance personnel in the form of graphical pages.

[0172] Service management platform: Provides functions for service registration, discovery, subscription, service on / off, service rate limiting, service degradation, and service weight adjustment.

[0173] Business Domain Open Platform: Provides developers with functions such as project creation, resource download, interface testing, and package upload.

[0174] 3. Microservice Application Model. The converged platform consists of multiple systems, mainly including: an automatic control system, an IoT system, intelligent bubble drainage, video surveillance, intelligent switch wells, and leak detection, etc. Each system is composed of multiple subsystems, modules, and components. For such a complex system, multiple application architecture views are used to display the system, first as a whole and then as parts, to maximize the display of the system's composition and how these systems collaborate to complete business functions.

[0175] (1) General-purpose components

[0176] Applications that need to be integrated are first horizontally split according to the generality of the components to form general service components for the platform.

[0177] (2) Business service components

[0178] First, the application is vertically split into modules based on business domains. These modules can be integrated and deployed within a single application, or deployed separately as individual modules or combinations of several modules. Then, each module is horizontally split, following a layered design based on interface programming standards, such as a business interface layer, business implementation layer, data access interface layer, data access implementation layer, and remote service invocation layer.

[0179] (3) Process Services

[0180] Define process services to implement business functions. When a process service depends on two or more atomic services, service orchestration is performed through a service orchestration framework. In the process service layer, a compensated TCC distributed transaction model is used to implement distributed transactions at the service layer. Process services register services and publish services externally, supporting at least HTTP / JSON. Process services can call each other.

[0181] (4) Service Interface

[0182] Throughout the architecture design process, an interface-oriented programming approach is adopted to implement the open / closed principle (open for extension, closed for modification) at the system architecture design level. When new functions are added to the system, it is not necessary to modify the existing system structure and code.

[0183] Taking a laser methane monitoring system as an example, such as Figure 3 As shown, through the unified fusion platform deployed in the regional dispatch center, the video surveillance module can directly view the live video of laser PTZ cameras at multiple production sites, eliminating the need to log into multiple laser PTZ monitoring systems. For example, it can display the number and name of laser methane PTZ cameras; read the IP address, username, and password of the laser methane PTZ cameras; read the real-time horizontal and vertical coordinates of the laser methane PTZ cameras; manually control the movement of the laser methane PTZ cameras; read and synchronously update the alarm threshold values ​​of the laser methane PTZ cameras (by simply modifying the alarm threshold values ​​in the laser methane PTZ camera client); and view the live video of the laser methane PTZ cameras, among other operations.

[0184] When the methane concentration at the production site exceeds the set methane concentration alarm threshold, the laser methane alarm service module can achieve real-time video linkage between the laser methane PTZ client at the site and the IoT client at the dispatch center. The IoT client will pop up a small video window showing the methane concentration alarm and capture and save the image. The pop-up video window displays the specific location of the methane concentration alarm and the real-time methane concentration value.

[0185] In addition, the data analysis service module can be used to perform data analysis and generate analysis views of the methane leak locations detected by the laser methane pan-tilt unit, such as... Figure 4All well sites can be viewed from the regional dispatch center using laser methane pan-tilt-zoom analysis; this allows for rapid identification of methane leak locations on the IoT analysis view, saving manual search time; and it enables continuous monitoring of whether the rectification of methane leaks at the production site has been completed.

[0186] In the converged platform, various functional modules exist independently as microservices, which can be independently developed, packaged, and deployed. Users can select and combine microservices according to their business needs to form application scenarios that conform to the business theme. For example, the laser methane monitoring described above forms a complete business application scenario from on-site monitoring, leak alarm, data analysis, and remote handling.

[0187] Those skilled in the art will understand that the above embodiments are specific examples of implementing the present invention, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of the present invention.

Claims

1. A method for fusing and processing real-time data from multi-component heterogeneous oil and gas production, characterized in that, Specifically, the following steps are included: S1: Unified collection of real-time production data from multiple independent systems; data is then entered into a database and stored through the device driver layer. S2: Unify and integrate the data from various systems, standardize the data encoding and storage, and standardize the data push to external systems; S3: The platform provides a unified data service interface to the outside world; S4: Integration of business functions.

2. The method as described in claim 1, characterized in that, Step 1 specifically includes the following steps: S101: Determine the data collection scope and method; establish a protocol driver layer by deploying platform collectors to uniformly connect to different communication protocols of various industrial equipment, IoT devices, video devices, etc. S1011: The independent system includes a production data SCADA system, a production video monitoring system, an intelligent bubble drainage system, an intelligent well switching system, and a leak detection system (laser pan-tilt unit). S1012: The data collection range for each independent system is as follows: Production video surveillance system: basic data, business data Intelligent bubble bath system: status information, real-time data Intelligent well control system: Equipment information Leakage Detection System (Laser Pan-Tilt-Zone): Equipment Information S1013: The communication protocol of the independent system corresponding to the protocol driver layer established by the platform collector is as follows: Production data SCADA system modbus-TCP Production video surveillance system TCP / IP Intelligent bubble extraction system Modbus-TCP Modbus-RTU Intelligent Switching Well System Leakage monitoring system (laser PTZ) Modbus-TCP S102: Design data acquisition functions based on the data acquisition scope and acquisition method of each independent system. Data Acquisition Driver Management: Supports common data acquisition protocols, including RESTful API, Webservice, Modbus, MQTT, RS485, CPCUA, and TCP / IP protocols; Data Collection Point Management: Configuration Protocol Management, Batch Import of Data Collection Points, Data Collection Point Maintenance Management Data simulator: Simulates data point creation and batch creation of simulation points. Video capture management: Configure video source name, alias, video type, address type, description, and URL information. Source data display: Display source data values ​​in a simple table format for easy real-time display during access, facilitating quick confirmation of the correctness of point source and point table configuration information. System configuration management: Configure data acquisition service information, set port number and UUID, set redundancy and breakpoint resume, and save information in device cache; System operation log: View user login and usage; view the operation status of collectors, drivers, and point sources; query by time and content.

3. The method as described in claim 1, characterized in that, In step S2, data governance, data standard management, and fusion processing are performed on the collected data from each independent system. The specific steps are as follows: S201: Data Cleaning Data cleaning addresses issues such as inconsistent, duplicate, missing, outdated, and uncorrelated data collection by performing unified cleaning. For real-time data, erroneous data is labeled and alerts are issued, and erroneous data is mapped to an "error database" for subsequent manual filtering and processing; For structured data, perform screening and alarms for erroneous data, duplicate data, and dead zone data, and process them according to rules; S202: Analysis and Calculation After cleaning, the data is processed again according to the calculation rules / model. The platform can calculate the corresponding data using batch calculation, statistical calculation, real-time calculation, and streaming calculation methods. S203: Data Quality Management Record the detection results and alarm results of abnormal data, manage the error information of the data in a targeted manner, identify and analyze the error information according to the code of the error data, and eliminate or modify the error information; S204: Data Security Data anonymization management: Sensitive information is transformed using defined anonymization rules to reliably protect sensitive and private data. Data encryption management: Setting encryption rules for critical information; Data review process management: Data modification requires relevant permission approval. The approval process for data modification can be configured according to business needs. S205: Data quality management.

4. The method as described in claim 1, characterized in that, In S3, the platform uniformly provides the following data service interfaces to external parties: Production Data SCADA System: OPC Production Video Surveillance System API Intelligent Bubble Extraction System OPC Intelligent Switching Well System OPC Leakage monitoring system (laser pan-tilt unit) OPC.

5. The method as described in claim 1, characterized in that, Step 4 also includes business function integration, specifically including the following steps: S401: Build a unified portal login engine on the production network side: realize enterprise portal functions through various application interfaces, single sign-on, and unified authentication methods, and provide users with a unified entry point for personalized access to various information and service resources; S402: The functional interfaces of the independent systems—production video monitoring system, intelligent bubble drainage system, intelligent well switching system, and leakage monitoring system (laser PTZ)—are transformed into microservices, with a unified data warehouse providing data services, forming a unified application platform on the production network side. This includes the following steps: S4021: Determine the integration method and scope of business functions; form functions for data query, display, monitoring and alarm, remote control, data analysis and decision information display, multi-dimensional topic analysis, and custom topic data mining; S4022: Building a Technical Framework for Business Function Integration (1) Service framework: Spring Cloud Alibaba was selected as the scaffolding for the microservice architecture solution; (2) Core Support Components Service Registry: Supports service discovery, dynamic configuration, service health monitoring, and service governance; a unified health check dashboard that can manage service availability and traffic based on health status; API Gateway: Supports dynamic routing rules and traffic control; It supports peak request flow control, allowing for the definition of various rate limiting rules; it supports recording request and response data, API latency analysis, and performance monitoring; it supports authentication, data masking, traffic cleaning, backend signing, API high availability, and a flexible fault tolerance mechanism. Configuration Center: Dynamic configuration services manage application and service configurations across all environments in a centralized, externalized, and dynamic manner. Dynamic configuration eliminates the need to redeploy applications and services when configurations change, making configuration management more efficient and agile. Centralized configuration management simplifies the implementation of stateless services and makes on-demand elastic scaling of services easier. (3) Service security A role-based authentication and authorization mechanism introduces digital certificates to sign and verify communication data, ensuring user identity authentication and the integrity of communication information. Digital signatures and timestamps guarantee the non-repudiation of business operations. (4) Other support services Console: Provides system management and parameter configuration functions for developers, and log level adjustment, cache update and viewing functions, task scheduling management functions, and abnormal task query functions for system administrators and maintenance personnel; Monitoring Center: Collects data sent by applications, performs statistics and analysis, and provides it to operations and maintenance personnel in the form of graphical pages; Service management platform: Provides functions for service registration, discovery, subscription, service on / off, service rate limiting, service degradation, and service weight adjustment; Business Domain Open Platform: Provides developers with functions for project creation, resource download, interface testing, and package upload. S403: Microservice applications, including (1) General-purpose components Applications that need to be integrated are first horizontally split according to the generality of the components to form general service components for the platform; (2) Business service components First, the application is vertically split into modules based on business domains. These modules can be integrated and deployed within a single application, or deployed separately as individual modules or combinations of several modules. Then, each module is horizontally split, following a layered design based on interface programming specifications, such as business interface layer, business implementation layer, data access interface layer, data access implementation layer, and remote service call. (3) Process Services Define process services to implement business functions; when a process service depends on two or more atomic services, service orchestration is performed through a service orchestration framework; in the process service layer, a compensated TCC distributed transaction model is used to implement distributed transactions in the service layer; process services register services and publish services to the outside world, supporting at least HTTP / JSON; process services can call each other; (4) Service Interface Throughout the architecture design process, an interface-oriented programming approach is adopted to implement the open / closed principle (open for extension, closed for modification) at the system architecture design level. When new functions are added to the system, it is not necessary to modify the existing system structure and code.

6. The method as described in claim 1, characterized in that, It achieves (1) data integration, (2) application integration, (3) enterprise information portal integration, and (4) data storage, analysis, and computing integration.

7. A computer device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images