Multi-modal oil reservoir data management method and system based on micro-service architecture
Through the multimodal reservoir data management method based on microservice architecture, the problem of difficulty in managing and processing large-scale complex reservoir data in the existing technology is solved, and efficient and reliable data management and rapid expansion are achieved.
Patent Information
- Application Number
- CN202411574537.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-06
- Publication Date
- 2025-02-11
AI Technical Summary
The prior art is difficult to effectively manage and process large-scale complex multimodal reservoir data, resulting in data source coupling and data consistency problems, and it is difficult to perform functional expansion under system construction.
The multimodal reservoir data management method based on the microservice architecture is adopted to compress and store multiple types of data through the reservoir data processing microservice set, and the reservoir data scheduling microservice set responds to data updates, quickly expands and integrates applications.
It realizes independent, real-time processing and integrated applications of multi-type reservoir data, improves the efficiency and reliability of data management, and supports fast access and functional expansion.
Smart Images
Figure CN119415292A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data twins, and in particular to a multimodal reservoir data management method and system based on a microservice architecture. Background Art
[0002] In modern oil and gas exploration and production, reservoir data plays a key role. These data include seismic data, geological models, well information, production data, etc., which are essential for decision-making, oilfield management and resource development. However, with the continuous increase in the scale of reservoir data and the diversification of data types, how to effectively store and manage this data has become a challenge.
[0003] Reservoir data refers to data used in the petroleum industry to describe and manage reservoir production, geology, engineering, etc. These data types are usually stored in digital form so that they can be analyzed, processed, and visualized by computer systems. The following are some common reservoir data types:
[0004] (1) Production data: including wellhead production data, bottom hole production data, output data, etc. These data describe the output, pressure, temperature and other information of oil wells, gas wells, water wells, etc.; (2) Geological data: including geological parameters such as formation structure, lithology, formation thickness, porosity, permeability, etc. These data are used to describe the geological characteristics of the reservoir and help evaluate the reserves and production potential of the reservoir. (3) Engineering data: including engineering parameters such as wellbore design, injection and production technology, well control equipment, pipeline layout, etc. These data are used to plan and manage the development and production process of the oil field; (4) Monitoring data: including production monitoring data, geological monitoring data, equipment monitoring data, etc. These data are used to monitor the production status of the oil field, the operating status of the equipment, and the geological changes of the reservoir. (5) Geographic information data: including map data, satellite images, terrain data, etc. These data are used to assist oil field exploration, production and management, and help understand the surface and underground conditions; (6) Real-time data: including real-time production data, real-time equipment status data, real-time seismic data, etc. These data are used to monitor and control the oil field production process in real time, helping to respond to production anomalies and emergencies in a timely manner.
[0005] It can be seen that the types of reservoir data are diverse, and there are great challenges in processing reservoir data. At present, the technologies available on the market are limited to the processing and storage of a single data type. For scenarios where different types of data are comprehensively applied, multiple data processing systems need to be connected separately. In a complex oil and gas digital twin system, different business systems and applications usually need to access various types of reservoir data. These reservoir data may come from different sources, such as geological, seismic, production and other systems. If each business system is directly connected to these reservoir data sources, data source coupling and data consistency problems will occur. In addition, different data processing systems are developed independently and may not be compatible with each other, and data access is not smooth, which will hinder the integrated application of multi-type data; in addition, each data source needs to be connected to a different data processing system separately, which has high construction costs, increases maintenance difficulties, and increases system redundancy, affecting system reliability; and under this system structure, it is difficult to expand functionality. Summary of the invention
[0006] The purpose of the present invention is to provide a multimodal reservoir data management method and system based on a microservice architecture to address all or part of the above-mentioned problems, so as to improve the efficiency and reliability of large-scale complex reservoir data management.
[0007] The technical solution adopted by the present invention is as follows:
[0008] A multimodal reservoir data management method based on a microservice architecture, comprising:
[0009] The reservoir data processing microservice set obtains the reservoir data updated by the reservoir data source;
[0010] The reservoir data processing microservice set compresses and stores text data, image data and video data in the reservoir data respectively;
[0011] In response to the update of the reservoir data, the reservoir data scheduling microservice set obtains the compressed reservoir data from the reservoir data processing microservice set and updates it to the corresponding business system.
[0012] Furthermore, the subscription service module of the reservoir data scheduling microservice set obtains information about updates to the reservoir data by subscribing to data update events of the reservoir data source.
[0013] Furthermore, when data is updated, the reservoir data source publishes the data update event to a specific topic of the message queue or a message routing agent exchange; the subscription service module of the reservoir data scheduling microservice set subscribes to the corresponding topic or exchange in the message queue to subscribe to the data update event of the reservoir data source.
[0014] Furthermore, there are one or more reservoir data sources, and each of the reservoir data sources publishes data update events to a corresponding topic of a message queue or a message routing agent exchange; and the subscription service module subscribes to one or more topics or exchanges in the message queue.
[0015] Furthermore, the distribution service module of the reservoir data scheduling microservice set monitors the data update notification of the subscription service module, and in response to the subscription service module receiving the data update notification indicating that the reservoir data has been updated, obtains the updated reservoir data of the reservoir data processing microservice set and distributes it to the corresponding business system.
[0016] Furthermore, the method for compressing the text data by the reservoir data processing microservice set includes:
[0017] Standardizing the text data;
[0018] The text data after the standardization process is encoded.
[0019] Furthermore, the method for encoding the text data includes:
[0020] Traversing the text data and counting the frequency of occurrence of each character;
[0021] Assign a code to each character based on its frequency of occurrence;
[0022] Replace each character in the text data with the corresponding code.
[0023] Furthermore, the method for compressing the image data by the reservoir data processing microservice set includes:
[0024] Performing color space conversion on the image data;
[0025] dividing the image data into a plurality of reservoir data blocks;
[0026] Performing frequency domain transformation on each of the reservoir data blocks to obtain corresponding high-frequency information and low-frequency information;
[0027] filtering out high frequency information in each of the reservoir data blocks;
[0028] All reservoir data blocks of the image data are combined.
[0029] Furthermore, the method for compressing the video data by the reservoir data processing microservice set includes:
[0030] Performing intra-frame compression on each frame of the video data;
[0031] For every two consecutive frame images, a differential image between the latter frame image and the former frame image is stored, and a motion vector of the differential image and a moving path of the same portion between the two consecutive frame images are stored.
[0032] The present invention also provides a multimodal reservoir data management system based on a microservice architecture, which includes a processor and a storage medium, wherein the storage medium stores a computer program, and the processor is connected to the storage medium to run the above-mentioned multimodal reservoir data management method based on the microservice architecture.
[0033] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:
[0034] The present invention uses a microservice architecture to process and store multiple types of reservoir data, realizing independent, real-time processing and integrated application of multiple types of reservoir data in the same system, improving the efficiency of centralized processing of multiple types of data, and improving the reliability of multi-type data management. Based on the microservice architecture, rapid access to multiple reservoir data sources under the same interface type is achieved without access barriers, and it can be rapidly expanded according to the update of the usage scenario. The present invention can flexibly subscribe to reservoir data according to needs for different business systems, and realize rapid expansion, which can better adapt to the ever-changing needs and environment, and has high practical value. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The present invention will now be described by way of example with reference to the accompanying drawings, in which:
[0036] Figure 1 It is a flow chart of the operation of the multimodal reservoir data management method.
[0037] Figure 2 It is a diagram of the operation architecture of the multimodal reservoir data management method. DETAILED DESCRIPTION
[0038] All features disclosed in this specification, or steps in all methods or processes disclosed, except mutually exclusive features and / or steps, can be combined in any manner.
[0039] Any feature disclosed in this specification (including any additional claims and abstract), unless otherwise stated, may be replaced by other equivalent or alternative features having similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.
[0040] In the existing reservoir digital twin system, the processing of reservoir data is usually only for a single data type, and the support for multi-modal reservoir data is relatively limited. This means that when processing different types of reservoir data, multiple resources and manpower need to be called, which also increases the complexity and management cost of the system. This may not achieve efficient processing of real-time reservoir data streams, thus affecting the efficiency and real-time performance of data processing.
[0041] Traditional reservoir data processing technologies lack flexible architecture design and are difficult to independently expand and maintain for different types of reservoir data, which limits the scalability and applicability of the system and presents certain difficulties in meeting large-scale reservoir data processing needs. In addition, these technologies are poorly integrated with the existing reservoir big data processing framework and cannot fully utilize the existing reservoir data processing platform and infrastructure. This leads to certain incompatibilities and integration difficulties in the reservoir data processing process, affecting the overall performance and stability of the system. The single type of data processing architecture determines that traditional reservoir data compression technology is difficult to expand and maintain modular functions, which increases the maintenance cost and complexity of the system and reduces the maintainability and stability of the system.
[0042] In order to meet the demand for efficient processing of large-scale multimodal reservoir data, the embodiment of the present application adopts a microservice architecture to design a management method for multi-type reservoir data, and uses a reservoir data processing microservice set to process multi-type reservoir data.
[0043] like Figure 1 As shown, an embodiment of the present application provides a multimodal reservoir data management method based on a microservice architecture, the method comprising the following process:
[0044] The reservoir data processing microservice set obtains the reservoir data updated by the reservoir data source;
[0045] The reservoir data processing microservice set compresses and stores the text data, image data, and video data in the (updated) reservoir data respectively;
[0046] The reservoir data scheduling microservice set responds to the update of the reservoir data, obtains the compressed reservoir data from the reservoir data processing microservice set and updates it to the corresponding business system.
[0047] Using the reservoir data processing microservice set to process multiple types of data has the following advantages:
[0048] 1. Data standardization: Different types of data can be converted and standardized into a unified format to ensure data consistency and availability.
[0049] 2. Data integration: It can integrate data sources from different systems, provide a unified data access portal, and simplify data access to each application module.
[0050] 3. Data analysis and application: It can provide targeted analysis and processing capabilities for different types of data to better support various oilfield application needs.
[0051] 4. Scalability and maintainability: The introduction of microservice set architecture can improve the scalability and maintainability of the system, making it easier to cope with the growth of future data scale and application requirements.
[0052] The following is an introduction to each of the above steps.
[0053] Step 1: The reservoir data processing microservice set obtains the reservoir data updated by the reservoir data source.
[0054] During the operation of the reservoir system, a large amount of data will be generated, such as production data, geological data, engineering data, etc., that is, a reservoir system is equivalent to a large reservoir data source, and different reservoir systems form multiple reservoir data sources. Before being applied to the business system, reservoir data needs to be preprocessed, and different types of data have different processing methods. Therefore, a large amount of reservoir data needs to be classified according to text data, image data, and video data types, and then the reservoir data is transferred to the reservoir data processing microservice for centralized processing. The processing of reservoir data is real-time, so the reservoir data source is to transfer the real-time updated reservoir data to the reservoir data processing microservice set, that is, the reservoir data processing microservice set obtains updated reservoir data.
[0055] Step 2: The reservoir data processing microservice set compresses and stores the text data, image data and video data in the reservoir data respectively.
[0056] like Figure 2 As shown, in the reservoir data processing microservice concentration, a text data processing module, an image data processing module and a video data processing module are designed for the three types of text data, image data and video data in the reservoir data. The three data processing modules compress the text data, image data and video data of the reservoir data respectively, and the three data processing modules are connected to the data storage module to store the compressed data.
[0057] In some embodiments, the method for compressing the text data includes:
[0058] Standardize text data;
[0059] Encode the standardized text data.
[0060] The so-called standardization processing is to standardize the text format for subsequent processing, which can include removing extra spaces, line breaks, tabs and other irrelevant characters.
[0061] After the standardization process, the encoding process of the text data is performed, in some embodiments, using an encoding method based on the frequency of occurrence of characters. According to an embodiment of the present application, the encoding process method includes:
[0062] Traversing the text data and counting the frequency of occurrence of each character;
[0063] Assign a code to each character based on its frequency of occurrence;
[0064] Replace each character in the text data with the corresponding encoding.
[0065] In this way, the length of the compressed text data is:
[0066] L(C)=∑ c∈C p(c)*l(c),
[0067] Where L(C) represents the total length after processing, p(c) represents the frequency of occurrence of character c, l(c) represents the length of character c after encoding, and C represents the character set in the text data.
[0068] In some embodiments, a method for compressing image data includes:
[0069] Perform color space conversion on image data;
[0070] dividing the image data into a plurality of reservoir data blocks;
[0071] Performing frequency domain transformation on each of the reservoir data blocks to obtain corresponding high-frequency information and low-frequency information;
[0072] filtering out high frequency information in each of the reservoir data blocks;
[0073] Combine all reservoir data blocks of the image data.
[0074] Typically, the image data generated is in RGB format, that is, each pixel is composed of three color channels: red, green, and blue, which are converted into brightness and chromaticity space to separate the brightness and speed information in each subsequent step.
[0075] After the image data is converted into color space, it is divided into multiple reservoir data blocks according to a set size. In this way, each reservoir data block consists of multiple pixels to form a local image area, which facilitates the processing of local features in the image data and reduces the computational complexity of the processing process.
[0076] For frequency domain transformation of the reservoir data block, in some embodiments, discrete cosine transform is used to obtain frequency domain coefficients of the reservoir data block:
[0077] R(u,v)=B(u)*B(v)*∑∑f(x,y)*cos[(2x+1)uπ]*cos[(2y+1)vπ],
[0078] Where R(u,v) represents the frequency domain coefficient, B(u) and B(v) are the normalized coefficients on the two chromaticity information, f(x,y) represents the pixel value at the pixel (x,y), and u and v represent the two chromaticity information at the pixel (x,y).
[0079] Through the above formula, for each reservoir data block, a set of frequency domain coefficients corresponding to each pixel can be obtained. Then, according to the frequency domain coefficients, the low-frequency information part of the image data is retained, and the high-frequency information part is filtered and discarded.
[0080] After filtering the high-frequency information of each reservoir data block of the image data, all its reservoir data blocks are combined and stored.
[0081] In some embodiments, a method for compressing video data includes:
[0082] Perform intra-frame compression on each frame of video data;
[0083] For every two consecutive frames, a differential image between the latter frame and the former frame is stored, and a motion vector of the differential image and a moving path of the former frame are stored, that is, inter-frame compression is performed on every two consecutive frames.
[0084] Specifically, first, each frame image is compressed within the frame. Then, the correlation between each two consecutive frames of images in the video data is used to further reduce the amount of all processed video data. By comparing the difference between the subsequent frame image and the previous frame image, the area in the frame image that moves or changes in the video data to be stored is determined, and the data storage module only stores the part that moves or changes. Therefore, when comparing two consecutive frames of images, the differential image between the subsequent frame image and the previous frame image is stored.
[0085] For the difference between two consecutive frames of images, in some embodiments, the difference is measured by the difference degree T:
[0086] T=∑∑|f(x,y)-f′(x+dx,y+dy)|,
[0087] In the formula, f(x,y) is the pixel value of pixel (x,y) in the frame image, and f′(x+dx,t+dy) represents the pixel value moved by the motion vector (dx,dy) with reference to the pixel (x,y) in the previous frame image.
[0088] Therefore, it is also necessary to store the motion vector (dx, dy) of the differential image in order to predict the pixel value of the subsequent frame image during video decoding.
[0089] In addition, for the same part of the two frames of images, the moving path of the same part between the two frames of images is found out through motion estimation and stored, so as to facilitate the prediction and restoration of the subsequent frame image during video decoding.
[0090] Through the above embodiments, compression processing and storage of updated reservoir data in each reservoir data source are achieved.
[0091] Step 3: In response to the update of the reservoir data, the reservoir data scheduling microservice set obtains the compressed reservoir data from the reservoir data processing microservice set and updates it to the corresponding business system.
[0092] The reservoir data dispatch microservice set is responsible for distributing the latest updated reservoir data to the business systems that require this reservoir data.
[0093] As to how to know whether the reservoir data has been updated, in some embodiments, the subscription service module of the reservoir data scheduling microservice set learns whether the reservoir data has been updated by subscribing to data update events of the reservoir data source.
[0094] Specifically, when data is updated, the reservoir data source publishes the data update event to a specific topic of the message queue or a message routing agent exchange. The subscription service module of the reservoir data scheduling microservice set subscribes to the corresponding topic or exchange in the message queue to subscribe to the data update event of the reservoir data source.
[0095] Usually, there can be one or more reservoir data sources, and all reservoir data sources publish data update events to the corresponding topics of the message queue or the message routing agent exchange. Similarly, there can be one or more subscription service modules, each of which is connected to the message queue and subscribes to one or more corresponding topics or exchanges in the message queue. When a reservoir data source publishes a data update event to the message queue, the subscription service module subscribed to the corresponding topic or exchange can receive a data update notification in real time, which indicates that the reservoir data has been updated.
[0096] After the subscription service module receives the data update notification, the reservoir data scheduling microservice set immediately obtains the updated reservoir data from the reservoir data processing microservice set for distribution.
[0097] In some embodiments, the reservoir data scheduling microservice set also includes a distribution service module. Once the subscription service module receives a data update notification, the distribution service module responds and obtains the updated reservoir data in the reservoir data processing microservice set and distributes it to the corresponding business system.
[0098] In the reservoir digital twin system, different business systems may need to obtain the same reservoir data. In some embodiments, the distribution service module distributes the acquired reservoir data to multiple business systems synchronously to ensure that the reservoir data is delivered to the required business systems in a timely manner. For example, the distribution service module monitors the subscription service module in real time, and in response to the subscription service module receiving a data update notification, it immediately obtains the updated reservoir data from the reservoir data processing microservice and distributes it to business systems such as production optimization, equipment monitoring, and predictive maintenance.
[0099] Different subscription service modules support subscription to different reservoir data sources, and different business systems can subscribe to different reservoir data sources through different subscription service modules. In some embodiments, distribution of reservoir data updated by different reservoir data sources is achieved by monitoring corresponding subscription service modules through different distribution service modules. According to the needs of the business system, the reservoir data source subscribed by the subscription service module is configured, and corresponding distribution service modules are configured for different subscription service modules, so that the reservoir data can be updated in real time to the subscribed business system.
[0100] In other embodiments, the subscription service module can immediately obtain the latest updated reservoir data storage module from the reservoir data processing microservice to its own business module in response to receiving the data update signal, and the corresponding distribution service module can obtain the latest updated reservoir data from the monitored subscription service module.
[0101] In other embodiments, the subscription service module still responds to receiving the data update signal and immediately obtains the latest updated reservoir data storage module from the reservoir data processing microservice set to its own business module. Different business systems still subscribe to the required reservoir data source according to the associated subscription service module. The reservoir data scheduling microservice set may also include a scheduling service module, which monitors all subscription service modules, and generates a corresponding data distribution notification when it monitors that a certain / certain subscription service modules receive a data update notification. The data distribution notification indicates the subscription service module and the corresponding business system that received the data update notification. The distribution service module has been changed. The distribution service module is changed to monitor the scheduling service module. After monitoring the data distribution notification of the scheduling service module, according to the instruction of the data distribution notification, the latest updated reservoir data is obtained from the data storage module or the subscription service module, and distributed to the business system indicated by the data distribution notification.
[0102] The present invention is not limited to the above-mentioned specific embodiments, but extends to any new features or any new combination disclosed in this specification, as well as any new method or process steps or any new combination disclosed.
Claims
1. A multimodal reservoir data management method based on microservice architecture, characterized in that: include: The reservoir data processing microservice set obtains the reservoir data updated by the reservoir data source; The reservoir data processing microservice set compresses and stores text data, image data and video data in the reservoir data respectively; In response to the update of the reservoir data, the reservoir data scheduling microservice set obtains the compressed reservoir data from the reservoir data processing microservice set and updates it to the corresponding business system.
2. The multimodal reservoir data management method based on microservice architecture according to claim 1, characterized in that: The subscription service module of the reservoir data scheduling microservice set obtains information about updates to the reservoir data by subscribing to data update events of the reservoir data source.
3. The multimodal reservoir data management method based on microservice architecture according to claim 2, characterized in that: When data is updated, the reservoir data source publishes the data update event to a specific topic of the message queue or a message routing agent exchange; the subscription service module of the reservoir data scheduling microservice set subscribes to the corresponding topic or exchange in the message queue to subscribe to the data update event of the reservoir data source.
4. The multimodal reservoir data management method based on microservice architecture according to claim 3, characterized in that: There are one or more reservoir data sources, and each of the reservoir data sources publishes data update events to a corresponding topic of a message queue or a message routing agent exchange; the subscription service module subscribes to one or more topics or exchanges in the message queue.
5. The multimodal reservoir data management method based on microservice architecture according to any one of claims 2 to 4, characterized in that: The distribution service module of the reservoir data scheduling microservice set monitors the data update notification of the subscription service module, and in response to the subscription service module receiving the data update notification indicating that the reservoir data has been updated, obtains the reservoir data updated in the reservoir data processing microservice set and distributes it to the corresponding business system.
6. The multimodal reservoir data management method based on microservice architecture according to claim 1, characterized in that: The method for compressing the text data by the reservoir data processing microservice set includes: Standardizing the text data; The text data after the standardization process is encoded.
7. The multimodal reservoir data management method based on microservice architecture according to claim 6, characterized in that: The method for encoding the text data includes: Traversing the text data and counting the frequency of occurrence of each character; Assign a code to each character based on its frequency of occurrence; Replace each character in the text data with the corresponding code.
8. The multimodal reservoir data management method based on microservice architecture according to claim 1, characterized in that: The method for compressing the image data by the reservoir data processing microservice set includes: Performing color space conversion on the image data; dividing the image data into a plurality of reservoir data blocks; Performing frequency domain transformation on each of the reservoir data blocks to obtain corresponding high-frequency information and low-frequency information; filtering out high frequency information in each of the reservoir data blocks; All reservoir data blocks of the image data are combined.
9. The multimodal reservoir data management method based on microservice architecture according to claim 1, characterized in that: The method for compressing the video data by the reservoir data processing microservice set includes: Performing intra-frame compression on each frame of the video data; For every two consecutive frame images, a differential image between the latter frame image and the former frame image is stored, and a motion vector of the differential image and a moving path of the same portion between the two consecutive frame images are stored.
10. A multimodal reservoir data management system based on microservice architecture, characterized in that: It includes a processor and a storage medium, the storage medium stores a computer program, the processor is connected to the storage medium, and runs the computer program in the storage medium to execute the multimodal reservoir data management method based on microservice architecture as described in any one of claims 1-9.