Online simulation real-time data processing method and equipment for coal bed gas gathering and transportation pipe network

Abstract

The invention discloses an online simulation real-time data processing method and equipment for a coal bed gas gathering and transportation pipe network, and the method comprises the steps: building one or more first sub-databases, so as to obtain target equipment information; establishing a total database, and initiating a data calling application to the first sub-database in a preset period; establishing an input database to store input information; based on a preset condition, judging whether the state and quality of the real-time data in the total database meet preset requirements or not; if not, comparing the total database with the first sub-database, determining missing content, and correcting the missing content by inputting the database; otherwise, storing the real-time data to a simulation database; and inputting the data in the simulation database into the gathering and transportation pipe network model for processing so as to assign the boundary conditions of the online simulation model. Therefore, the accuracy of online simulation real-time data is effectively guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of online simulation of coalbed methane gathering and transportation pipeline network, and in particular to a real-time data processing method and equipment for online simulation of coalbed methane gathering and transportation pipeline network. Background Art

[0002] With the continuous development of coalbed methane wells, the types and quantities of real-time data collected by monitoring instruments in its gathering and transportation pipeline network system are also increasing. These real-time data include instantaneous gas production, gas production pressure and gas production temperature collected from wellheads and gas gathering stations. As a result, the online simulation of the pipeline network requires comprehensive, fast and effective reading of the real-time data of each instrument in the coalbed methane gathering and transportation pipeline network, so as to avoid abnormalities in the quality and status of the real-time data screen (i.e. missing, abnormal and not updated). Therefore, ensuring the speed and accuracy of online simulation calculations is an important problem that must be solved in the online simulation of coalbed methane gathering and transportation pipeline networks. Summary of the invention

[0003] The embodiment of the present invention at least partially solves the technical problem of abnormal quality and status of online simulation real-time data of coalbed methane gathering and transportation pipeline network in the prior art by providing a method and equipment for processing online simulation real-time data of coalbed methane gathering and transportation pipeline network, and achieves the technical effect of improving the speed and accuracy of online simulation calculation.

[0004] In the first aspect, to solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0005] A method for processing real-time data of online simulation of a coalbed methane gathering and transportation pipeline network, comprising:

[0006] Establish one or more first sub-databases to obtain target device information; establish a general database to initiate data call applications to the above-mentioned first sub-databases at a preset period; establish an input database to store input information;

[0007] Based on the preset conditions, determine whether the state and quality of the real-time data in the above-mentioned total database meet the preset requirements; if not, compare the above-mentioned total database with the above-mentioned first sub-database to determine the missing content, and correct the above-mentioned missing content through the above-mentioned input database; otherwise, store the above-mentioned real-time data in the simulation database;

[0008] The data in the above simulation database is input into the gathering and transportation pipeline network model for processing to assign values ​​to the boundary conditions of the online simulation model.

[0009] Optionally, when the first sub-database and the main database are located in different systems, the method further includes:

[0010] In the system where the above-mentioned general database is located, multiple second sub-databases are established, each of the above-mentioned second sub-databases corresponds to one of the above-mentioned first sub-databases, and any of the above-mentioned second sub-databases initiates a data call application to the corresponding above-mentioned first sub-database at a preset period; the above-mentioned general database calls the real-time data of all the above-mentioned second sub-databases;

[0011] Based on the preset conditions, determine whether the state and quality of the real-time data in the above-mentioned total database meet the preset requirements; if not, compare the corresponding first sub-database and the above-mentioned second sub-database to determine the missing content, and correct the missing content through the above-mentioned input database; otherwise, store the above-mentioned real-time data in the simulation database;

[0012] The data in the above simulation database is input into the gathering and transportation pipeline network model for processing to assign values ​​to the boundary conditions of the online simulation model.

[0013] Optionally, before determining whether the state and quality of the real-time data in the total database meet preset requirements, the method further includes:

[0014] An intermediate database is set up, and the intermediate database filters out real-time data of a preset type from the total database, and proceeds to the next step based on the real-time data.

[0015] Optionally, the step of determining whether the quality of the real-time data in the total database meets preset requirements specifically includes:

[0016] Determine whether the real-time data in the above-mentioned total database is complete;

[0017] If so, the real-time data is added to the first task queue; otherwise, the real-time data in the first sub-database and the corresponding second sub-database are compared to determine the missing device, read the missing data corresponding to the missing device in the entry database, and add the missing data and the data in the total database to the first task queue.

[0018] Optionally, the step of adding the missing data and the data in the total database to the first task queue specifically includes:

[0019] The storage data of each cache level is used as a data source, and the missing data is added to the first task queue in an asynchronous message manner.

[0020] Optionally, the step of determining whether the quality of the real-time data in the total database meets preset requirements specifically includes:

[0021] Determine whether the real-time data in the first task queue is valid;

[0022] If yes, then add the above real-time data to the second task queue;

[0023] On the contrary, invalid data is classified according to data type, and corresponding preset programs are called according to different categories for assignment; based on the above-mentioned input database, the accuracy of the assigned real-time data is checked; the checked real-time data is added to the preset position of the second task queue.

[0024] Optionally, the step of determining whether the state of the real-time data in the above-mentioned total database meets the preset requirements specifically includes:

[0025] Determine whether the real-time data in the second task queue is updated;

[0026] If so, the real-time data is stored in the simulation database;

[0027] On the contrary, according to different non-updated indicators, the above real-time data is periodically checked by replacing the corresponding data read from the input database or predicting using a sliding window; the checked real-time data is stored in the simulation database.

[0028] Optionally, before inputting the data in the simulation database into the gathering and transportation pipeline network model for processing, the method further includes: publishing the real-time data of the simulation database to a display terminal through a communication interface.

[0029] Optionally, the step of inputting the data into the gathering and transportation pipeline network model for processing specifically includes: processing different types of real-time data in the simulation database by averaging or summing the data according to the production and operation characteristics of the nodes in the gathering and transportation pipeline network.

[0030] In a second aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps corresponding to the method of the first aspect when executing the computer program.

[0031] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0032] In an embodiment of the present invention, the first sub-database is used as a benchmark reference to compare the quality and status of the real-time data of the total database and determine the abnormal data. The abnormal data is then corrected using the input data, thereby effectively ensuring the accuracy of the real-time data of the online simulation. Data caching and updating are performed in a specified cycle and task queue manner to promptly alleviate the state of real-time data reading and writing lag caused by the large amount of real-time data and high update frequency. This alleviates the bandwidth pressure caused by the Internet of Things instruments of the coalbed methane gathering and transportation pipeline network collecting a large amount of real-time data and transmitting it to the pipeline simulation, thereby improving the reading speed of the effective data type of the online simulation. While ensuring data accuracy, the stability of the online simulation calculation process is improved, ensuring the long-term and stable operation of the online simulation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 A structural block diagram of a coalbed methane gathering and transportation pipeline network online simulation real-time data processing method provided by the present invention;

[0035] Figure 2 A flow chart of a method for online simulation of real-time data processing of a coalbed methane gathering and transportation pipeline network provided by the present invention;

[0036] Figure 3 A flowchart for determining whether real-time data is complete in the present invention;

[0037] Figure 4 A flowchart for determining whether real-time data is valid in the present invention;

[0038] Figure 5 This is a flow chart for determining whether real-time data is updated in the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0040] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0042] In the description of the present invention, it should also be noted that, unless otherwise clearly specified and limited, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0043] It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of the present application, rather than limitations of the technical solutions of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0044] The technical solution of the embodiment of the present invention is to solve the above technical problems, and the overall idea is as follows:

[0045] The first sub-database is used as a reference to compare the real-time data quality and status of the total database to determine abnormal data, and then the input data is used to correct the abnormal data to avoid abnormal quality and status of the real-time data.

[0046] Embodiment 1:

[0047] In an embodiment of the present invention, there is provided Figure 1 and Figure 2 A method for processing real-time data of online simulation of a coalbed methane gathering and transportation pipeline network is shown, the method comprising steps S101 to S103:

[0048] Step S101, establishing one or more first sub-databases to obtain target device information; establishing a general database to initiate a data call application to the first sub-database at a preset period; establishing an entry database to store entry information;

[0049] It should be noted that this embodiment is applied to the real-time data acquisition of the monitoring instrument of the gathering and transportation pipeline network in the same system. Among them, the first sub-database is used to directly obtain various real-time data from the field instrument (target device). The real-time data includes instantaneous gas production, gas production pressure, gas production temperature and the corresponding instrument identity (hereinafter referred to as ID), etc. Because the instrument data contained in the entire gathering and transportation pipeline network is too large, multiple first sub-database partitions are required to obtain the instrument information of different blocks, and the total database is used to collect the real-time information of all first sub-databases. In addition, the instrument monitoring of the gathering and transportation pipeline network will also have manual data meter reading and summarize it into statistical reports. Therefore, an entry database is set to summarize all statistical reports. When the first sub-database total database initiates a data call application to the first sub-database at a preset period, that is, the application cycle for the first sub-database is 5min / time, and the application cycle for the entry database is 1 day / time.

[0050] Step S102, based on preset conditions, determine whether the status and quality of the real-time data in the total database meet the preset requirements; if not, compare the total database with the first sub-database to determine the missing content, and correct the missing content by entering the database; otherwise, store the real-time data in the simulation database;

[0051] It should be noted that the first sub-database is set locally in the block, while the total database collects data from each block. Due to the continuous operation of a large number of instruments in the gathering and transportation network, the system hardware may be damaged due to aging, poor line contact or network disconnection, and the system software may also crash due to hardware compatibility, poor CPU, memory and other performance, resulting in data corruption or loss, affecting data quality and update status. Therefore, this embodiment uses a comparison between the total database and the first sub-database to obtain missing data types.

[0052] Before judging whether the status and quality of the real-time data in the total database meet the preset requirements, the present embodiment sets up an intermediate database, which filters out the real-time data of preset types from the total database. The parameters required for the simulation are filtered out, including pressure, temperature and output. The filtered real-time data are re-sorted according to their IDs to facilitate querying their gas production pressure, instantaneous gas production and gas production temperature labels. Then the following judgment is made based on the real-time data after the above processing.

[0053] Conduct real-time data quality assessment:

[0054] Specifically, when judging whether the real-time data in the total database is complete, the judgment method can be to check whether its syntax, data type, value range and data structure meet the expected specifications. For example, check whether the data contains necessary fields, including instrument ID, temperature, pressure and gas production, and whether the types of these fields are correct and whether the lengths meet the requirements. Among them, the technical solution for judging whether the data is complete can be selected and optimized as needed, and is not limited here.

[0055] If the judgment result is that the real-time data is complete, the real-time data will be added to the first task queue and the next step of processing will be entered; otherwise, the real-time data in the main database and the first sub-database will be compared to determine the missing device, read the missing data corresponding to the missing device in the entry database, and add the missing data and the data in the main database to the first task queue.

[0056] It should be noted that the comparison between the main database and the first sub-database is intended to ensure the accuracy of data transmitted from the first sub-database to the main database. That is, when some data is lost, the first sub-database is used as a benchmark to check the real-time data of the main database to find the missing instruments, and the real-time data corresponding to the missing instruments in the input database are used to supplement them. Specifically, the storage data of each cache level is used as the data source, and the missing data is added to the first task queue based on asynchronous messages.

[0057] That is to say, when the system completes a transaction, it encapsulates the transaction result and related information into a message. The system sends this message to the message queue, and a special service obtains the message from the message queue. This service performs corresponding operations according to the transaction result: if the transaction is successful, it only needs to discard the message; if the transaction fails, it needs to perform compensation operations, and the specific compensation operations are to supplement the missing data content as mentioned above. That is, by separating transaction processing and compensation operations, they can be processed in parallel, thereby improving overall performance. And using message queues for asynchronous communication can decouple the system and compensation services in time, thereby reducing the complexity of the system. Therefore, through asynchronous message processing, real-time transaction processing and compensation operations are realized, improving the real-time performance of the system.

[0058] Then, it is determined whether the real-time data in the first task queue is valid. The specific determination method is: firstly establish the rules or regular expressions of the real-time data, and then regard the data that does not meet these rules or regular expressions as invalid data. For example, in the rules of numerical data, the output data cannot be negative. If the output data is negative, it can be determined that the data is invalid.

[0059] If the judgment result is valid, the real-time data will be added to the second task queue and enter the next processing flow; otherwise, the invalid data will be classified according to the data type, and the corresponding preset program will be called according to the different categories for assignment; based on the input database, the accuracy of the assigned real-time data will be checked; the checked real-time data will be added to the preset position of the second task queue.

[0060] Among them, during the transmission of real-time data in various links, a single node may occasionally have abnormal states, such as null values, negative values, 0, maximum values, non-numeric values, etc. These abnormal data do not meet the simulation boundary requirements. Directly assigning values ​​to simulation nodes as boundary conditions will cause system crashes and affect long-term stable operation. According to the characteristics of abnormal data, corresponding methods are formulated for reassignment. The purpose is to ensure that the real-time data does not deviate from the actual situation while ensuring the stable operation of the online simulation. For details, please refer to Figure 4 , data classification uses different verification methods for different abnormal data, uses the range limit value assignment method for negative values, 0 values ​​and maximum values, and uses the nearest assignment method for null values ​​and non-numeric values; after completing the assignment, the single real-time data verification accuracy is obtained based on whether the error between the total database and the first sub-database is within the preset range. The real-time data that meets the required accuracy is input into the next step for status detection.

[0061] To judge the real-time data status:

[0062] Specifically, it is determined whether the real-time data in the second task queue is updated by detecting the length of time during which the real-time data is not updated.

[0063] If it is judged to have been updated, the real-time data will be stored in the simulation database; otherwise, according to different non-updated indicators, the real-time data will be periodically checked by replacing the corresponding data in the entry database or predicting with a sliding window; the checked real-time data will be stored in the simulation database.

[0064] It should be noted that when a single node or all real-time data is interrupted for a short period of time during the communication process, the online simulation will receive duplicate data at a specified frequency, which is meaningless. The dead data is judged and corrected by moving the window prediction data assignment. The purpose is to ensure that the online simulation boundary is in a timely changing state in the short term, so as to buy time for the communication to resume normal.

[0065] For details, please refer to Figure 5Different processing methods are used for different non-updated time periods. When the real-time data has not been updated for more than or equal to 10 times, the sliding window prediction method is used to supplement the data; when the real-time data has not been updated for more than or equal to 24 hours, the corresponding data entered into the database is read and supplemented; when the real-time data entered into the database has not been updated for more than or equal to 24 hours, the sliding window prediction method is used to supplement the data. Finally, the supplemented data is stored in the simulation database.

[0066] Step S103, inputting the data in the simulation database into the gathering and transportation network model for processing to assign values ​​to the boundary conditions of the online simulation model.

[0067] It should be noted that the gathering and transportation network model defines the nodes of the coalbed methane pipeline network according to the actual working conditions. According to the production and operation characteristics of the nodes in the gathering and transportation pipeline network, different types of real-time data in the simulation database are processed by averaging or summing.

[0068] In detail, taking flow data as an example, when a certain main node is in actual working conditions, it is necessary to sum up the flow of multiple instruments, and then use the production and operation characteristics of different nodes to sum up the flow data, average the pressure and temperature data, and other processing operations, so as to re-assign the value to make up for the problem of no unified measurement of key nodes on site.

[0069] Embodiment 2:

[0070] Furthermore, when the first sub-database and the main database are located in different systems, the method further includes:

[0071] In the system where the main database is located, multiple second sub-databases are established, each second sub-database corresponds to a first sub-database, and any second sub-database initiates a data call application to the corresponding first sub-database at a preset period; the main database calls the real-time data of all second sub-databases;

[0072] It should be noted that the difference between this embodiment and embodiment 1 is that this embodiment is applied to a system where the main database and the first sub-database are not in the same system, and the main database collects real-time data of the first sub-databases in multiple different systems. At this time, in order to facilitate comparison and separate transmission, a number of second sub-databases corresponding to the number of first databases is set on the system where the main database is located. The remaining steps are the same as those in embodiment 1.

[0073] Based on the preset conditions, determine whether the status and quality of the real-time data in the total database meet the preset requirements; if not, compare the corresponding first sub-database and the second sub-database to determine the missing content, and correct the missing content by entering the database; otherwise, store the real-time data in the simulation database;

[0074] It should be noted that, unlike Example 1, Figure 3 In this embodiment, the first sub-database and the second sub-database are used for comparison. Since all the first sub-databases are correspondingly provided with a second sub-database, the comparison can be performed in multiple lines at the same time, thereby improving the comparison efficiency. The remaining steps are the same as those in Embodiment 1.

[0075] The data in the simulation database is input into the gathering and transportation pipeline network model for processing to assign values ​​to the boundary conditions of the online simulation model.

[0076] It should be noted that the gathering and transportation network model defines the nodes of the coalbed methane pipeline network according to the actual working conditions. According to the production and operation characteristics of the nodes in the gathering and transportation pipeline network, different types of real-time data in the simulation database are processed by averaging or summing.

[0077] In detail, taking flow data as an example, when a certain main node is in actual working conditions, it is necessary to sum up the flow of multiple instruments, and then use the production and operation characteristics of different nodes to sum up the flow data, average the pressure and temperature data, and other processing operations, so as to re-assign the value to make up for the problem of no unified measurement of key nodes on site.

[0078] Furthermore, before the data in the simulation database is input into the gathering and transportation network model for processing, the method also includes: publishing the real-time data of the simulation database to the display terminal through the communication interface. Thus, the overall integrity, validity and timeliness of the real-time data are displayed according to the number of tags, the relative error between the total gas production and the total external transmission, and the data update cycle.

[0079] Embodiment 3:

[0080] Based on the same inventive concept, an embodiment of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the online simulation real-time data processing method of Embodiment 1 and / or Embodiment 2 is implemented.

[0081] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time data processing method for online simulation of coalbed methane gathering and transportation pipeline network, characterized in that: The method comprises: Establish one or more first sub-databases to obtain target device information; establish a general database to initiate data call applications to the first sub-databases at a preset period; establish an entry database to store entry information; Based on the preset conditions, determine whether the status and quality of the real-time data in the total database meet the preset requirements; if not, compare the total database with the first sub-database to determine the missing content, and correct the missing content through the input database; otherwise, store the real-time data in the simulation database; The data in the simulation database is input into the gathering and transportation pipeline network model for processing to assign values ​​to the boundary conditions of the online simulation model.

2. The method according to claim 1, characterized in that When the first sub-database and the main database are located in different systems, the method further includes: In the system where the general database is located, multiple second sub-databases are established, each of which corresponds to one first sub-database, and any second sub-database initiates a data call application to the corresponding first sub-database at a preset period; the general database calls the real-time data of all the second sub-databases; Based on the preset conditions, determine whether the state and quality of the real-time data in the total database meet the preset requirements; if not, compare the corresponding first sub-database and the second sub-database to determine the missing content, and correct the missing content through the input database; otherwise, store the real-time data in the simulation database; The data in the simulation database is input into the gathering and transportation pipeline network model for processing to assign values ​​to the boundary conditions of the online simulation model.

3. The method according to claim 2, characterized in that Before determining whether the state and quality of the real-time data in the total database meet the preset requirements, the method further includes: An intermediate database is set up, and the intermediate database filters out real-time data of a preset type from the total database, and proceeds to the next step based on the real-time data.

4. The method according to any one of claims 2 to 3, characterized in that: The step of judging whether the quality of the real-time data in the total database meets the preset requirements specifically includes: Determining whether the real-time data in the total database is complete; If so, the real-time data is added to the first task queue; otherwise, the real-time data in the first sub-database and the corresponding second sub-database are compared to determine the missing device, the missing data corresponding to the missing device in the entry database is read, and the missing data and the data in the total database are added to the first task queue.

5. The method according to claim 4, characterized in that The step of adding the missing data and the data of the total database to the first task queue specifically includes: The storage data of each cache level is used as a data source, and the missing data is added to the first task queue by asynchronous messaging.

6. The method according to claim 4, characterized in that The step of judging whether the quality of the real-time data in the total database meets the preset requirements specifically includes: Determine whether the real-time data in the first task queue is valid; If yes, adding the real-time data to the second task queue; On the contrary, invalid data is classified according to data type, and corresponding preset programs are called according to different categories for assignment; based on the input database, the accuracy of the assigned real-time data is checked; and the checked real-time data is added to the preset position of the second task queue.

7. The method according to claim 5, characterized in that The step of judging whether the state of the real-time data in the total database meets the preset requirements specifically includes: Determine whether the real-time data in the second task queue is updated; If so, storing the real-time data in the simulation database; On the contrary, according to different non-updated indicators, the real-time data is periodically checked by replacing the corresponding data read from the database or predicting using a sliding window; the checked real-time data is stored in the simulation database.

8. The method according to claim 1, characterized in that Before inputting the data in the simulation database into the gathering and transportation pipeline network model for processing, the method further includes: The real-time data of the simulation database is published to the display terminal through the communication interface.

9. The method according to claim 1, characterized in that The step of inputting the data to the gathering and transportation network model for processing specifically includes: According to the production and operation characteristics of the nodes in the gathering and transportation pipeline network, different types of real-time data in the simulation database are processed in an averaging or summing manner.

10. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method steps described in any one of claims 1 to 9 when executing the computer program.

Images