Pipeline data detection method, system, device and storage medium

Abstract

The application discloses a pipeline data detection method, system, device and storage medium, relates to the technical field of pipeline data detection, and aims to solve the problem of low accuracy of pipeline data detection and the problem of being unable to accurately identify abnormal data. The method comprises the following steps: acquiring sampling data of a plurality of data collection points of a pipeline to be detected, the sampling data of each data collection point being time series data, and the sampling data being used for representing the operation characteristics of the pipeline to be detected; performing data anomaly determination on the sampling data to obtain an anomaly determination result of the sampling data; the data anomaly determination comprises integrity determination and / or reliability determination; the integrity determination comprises at least one of the following: packet loss data determination, missing data determination and garbled data determination; the reliability determination is used for determining error data in the sampling data; and a data detection result of the pipeline to be detected is determined according to the anomaly determination result, an error log is generated, and an alarm is given.

Description

Technical Field

[0001] This application relates to the field of pipeline data detection technology, and in particular to a pipeline data detection method, system, equipment and storage medium. Background Technology

[0002] Currently, oil and gas pipelines are equipped with a considerable number of detection devices (such as temperature, flow, and pressure sensors) and are able to upload the collected data to the main control center. However, during the data acquisition and uploading process, situations such as sensor failure, communication failure, packet loss, and illegal attacks may occur, making it impossible for the main control center to obtain the correct data from the pipeline.

[0003] Currently, missing, out-of-range, and limit-exceeding data can be identified, and the relationship between strongly correlated physical quantities can be used to determine whether the data has been tampered with or transmitted incorrectly. However, this method can only handle relatively simple tampering methods (tampering with individual data). For more covert and complex tampering methods, it is difficult to identify whether data has been tampered with. For example, an attacker might systematically adjust all data in a data transmission to make it consistent while maintaining physical relationships. This operation usually makes the tampered data appear to conform to physical laws, thus evading anomaly detection methods.

[0004] Therefore, the current accuracy of pipeline data detection is low, and it is unable to accurately identify data with anomalies. Summary of the Invention

[0005] The purpose of this application is to provide a pipeline data detection method, system, device and storage medium, which aims to solve the problem of low accuracy in pipeline data detection and inability to accurately identify abnormal data.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] Firstly, this application provides a pipeline data detection method, including:

[0008] The sampling data of multiple data acquisition points of the pipeline to be tested are acquired. The sampling data of each data acquisition point is time series data. The sampling data is used to characterize the operating characteristics of the pipeline to be tested.

[0009] The sampled data is subjected to data anomaly detection to obtain the anomaly detection results; the data anomaly detection includes: integrity detection and / or reliability detection; the integrity detection includes at least one of the following: packet loss detection, missing data detection, and garbled data detection; the reliability detection is used to determine the erroneous data in the sampled data;

[0010] Based on the anomaly detection results, determine the data detection results of the pipeline to be tested, generate error logs, and issue alarms.

[0011] The pipeline data detection method provided in this application determines the integrity and / or reliability of the sampled data from multiple data collection points of the pipeline under test by performing data anomaly determination on the sampled data. Specifically, it determines whether the sampled data from multiple data collection points of the pipeline under test includes lost data, missing data, and garbled data, and whether there is erroneous data in the sampled data, thereby obtaining the anomaly determination result of the sampled data. Based on the anomaly determination result, it determines the data detection result of the pipeline under test, generates an error log, and issues an alarm.

[0012] In this way, by analyzing and judging the sampled data from multiple data collection points of the pipeline under inspection, it is possible to determine whether there is packet loss, missing data, garbled data, or erroneous data in the sampled data. Based on the judgment results, the data inspection result of the pipeline under inspection is determined, and an error log is generated and an alarm is issued to provide a prompt. This improves the accuracy of pipeline data inspection, accurately identifies abnormal data, effectively reduces the probability of pipeline failure, and ensures the safe operation of the pipeline.

[0013] In some embodiments, where data anomaly determination includes integrity determination, and integrity determination includes packet loss determination, missing data determination, and garbled data determination, data anomaly determination is performed on the sampled data to obtain anomaly determination results for the sampled data, including:

[0014] Regarding the sampled data from the first data acquisition point, if the main control center does not receive the data transmitted by the first data acquisition point at the data transmission time point, the anomaly determination result is that there is packet loss in the sampled data of the first data acquisition point. The first data acquisition point is any one of multiple data acquisition points, and the data transmission time point is the time when the data acquisition point transmits data to the main control center.

[0015] When the data transmitted from the first data acquisition point received by the main control center fails the data bit verification, the anomaly determination result is that there is missing data in the sampled data of the first data acquisition point;

[0016] When the data transmitted from the first data acquisition point received by the main control center fails the data error correction verification, the anomaly determination result is that there is garbled data in the sampled data of the first data acquisition point.

[0017] In some embodiments, the sampling data includes at least one of the following: flow rate data, temperature data, pressure data, and density data, and the sampling data of each of the multiple data collection points includes data within a target time period;

[0018] In cases where data anomaly detection includes credibility assessment, the sampled data is subjected to data anomaly detection to obtain the anomaly detection results, including:

[0019] The target data from the sampling data of the second data acquisition point and the sampling data of at least one third data acquisition point are input into the discrimination model to obtain the predicted data of the target data of the second data acquisition point at the target time. The at least one third data acquisition point is at least one data acquisition point that is adjacent to the second data acquisition point among multiple data acquisition points. The target data is any one of flow data, temperature data, pressure data, and density data. The target time is the last time in the target time period.

[0020] When the difference between the predicted data and the target data at the second data acquisition point at the target time exceeds a preset threshold, the anomaly determination result is that the sampled data of at least one of the second data acquisition point and at least one third data acquisition point includes abnormal data.

[0021] In some embodiments, the sampled data of at least one of the data acquisition points, including the second data acquisition point and at least one third data acquisition point, whose anomaly determination result is determined to be abnormal data, includes:

[0022] For the fourth data acquisition point, two data acquisition point groups adjacent to the fourth data acquisition point are determined. The fourth data acquisition point is any one of the second data acquisition point and at least one third data acquisition point. The two data acquisition point groups include different data acquisition points.

[0023] For any one of the two data acquisition point groups, the sampled data of the fourth data acquisition point, as well as the target data in the sampled data of each data acquisition point included in any data acquisition point group, are input into the discrimination model to obtain the predicted data of the target data of the fourth data acquisition point at the target time.

[0024] When the difference between the predicted data determined based on each of the two data acquisition point groups and the target data of the fourth data acquisition point at the target time exceeds a preset threshold, the anomaly determination result is that the sampled data of the fourth data acquisition point includes abnormal data.

[0025] In some embodiments, the discrimination model includes: a gated recurrent unit (GRU), a convolutional neural network (CNN), and an artificial neural network (ANN). The GRU is used to extract the temporal features of the data, and the CNN is used to extract the spatial features of the data.

[0026] The target data from the sampling data of the second data acquisition point and the sampling data of at least one third data acquisition point are input into the discrimination model to obtain the predicted data of the target data of the second data acquisition point at the target time, including:

[0027] The target data from the sampled data of the second data acquisition point and the sampled data of at least one third data acquisition point are input into the gated recurrent unit (GRU) to obtain the timing characteristics.

[0028] The target data from the sampling data of the second data acquisition point and the sampling data of at least one third data acquisition point are input into the convolutional neural network (CNN) to obtain spatial features.

[0029] By inputting temporal and spatial features into an artificial neural network (ANN), the predicted data of the target data at the second data acquisition point at the target time is obtained.

[0030] In some embodiments, where data anomaly determination includes credibility determination, data anomaly determination is performed on the sampled data to obtain anomaly determination results for the sampled data, including:

[0031] For any data collection point among multiple data collection points, if the sampled data of any data collection point exceeds the reference data threshold, the anomaly determination result is that there is erroneous data in the sampled data of that data collection point.

[0032] In some embodiments, acquiring sampling data from multiple data acquisition points of the pipeline to be inspected includes:

[0033] Acquire sensor data collected by sensors at multiple data acquisition points on the pipeline to be inspected;

[0034] Data preprocessing is performed on the sensor data to obtain the sampling data of each data acquisition point among multiple data acquisition points. Data preprocessing includes at least one of the following: adding timestamps, data normalization, and data cleaning.

[0035] Secondly, this application provides a pipeline data detection system, comprising:

[0036] Data acquisition module, anomaly detection module, and anomaly alarm module;

[0037] The data acquisition module is configured to acquire sampling data from multiple data acquisition points of the pipeline under test. The sampling data from each data acquisition point is time-series data, which is used to characterize the operating characteristics of the pipeline under test.

[0038] The anomaly detection module is configured to perform data anomaly detection on the sampled data to obtain the anomaly detection result of the sampled data; the data anomaly detection includes: integrity detection and / or credibility detection; the integrity detection includes at least one of the following: packet loss detection, missing data detection, and garbled data detection; the credibility detection is used to determine the erroneous data in the sampled data;

[0039] The anomaly alarm module is configured to determine the data detection result of the pipeline to be detected based on the anomaly judgment result, generate error logs and issue alarms.

[0040] Thirdly, this application provides a pipeline data detection device, comprising:

[0041] The acquisition unit is used to acquire sampling data from multiple data acquisition points of the pipeline under test. The sampling data from each data acquisition point is time-series data, which is used to characterize the operating characteristics of the pipeline under test.

[0042] The judgment unit is used to perform data anomaly judgment on the sampled data to obtain the anomaly judgment result of the sampled data; the data anomaly judgment includes: integrity judgment and / or credibility judgment; the integrity judgment includes at least one of the following: packet loss judgment, missing data judgment, and garbled data judgment; the credibility judgment is used to determine the erroneous data in the sampled data;

[0043] The processing unit is used to determine the data detection results of the pipeline to be detected based on the anomaly judgment results, generate error logs and issue alarms.

[0044] Fourthly, this application provides a pipeline data detection device, including: a processor and a memory; wherein the memory is used to store one or more programs, the one or more programs including computer execution instructions, and when the pipeline data detection device is running, the processor executes the computer execution instructions stored in the memory to cause the pipeline data detection device to perform the pipeline data detection method of the first aspect.

[0045] The pipeline data detection device can be a network device or a component of a network device, such as a chip system within the network device. This chip system supports the network device in implementing the functions involved in any of the possible implementations described above, such as data acquisition, processing, correction, prediction, or early warning. The chip system includes chips, but may also include other discrete devices or circuit structures.

[0046] Fifthly, this application provides a computer-readable storage medium in which computer-executable instructions stored in the computer-readable storage medium are executed by a processor of a pipeline data detection device, enabling the pipeline data detection device to perform the pipeline data detection method of the first aspect.

[0047] Sixthly, this application provides a computer program product, which includes a computer program or instructions that, when executed on a computer, cause the computer to perform the pipeline data detection method of the first aspect.

[0048] It should be noted that the aforementioned computer program or instructions may be stored, in whole or in part, on a computer-readable storage medium. This computer-readable storage medium may be packaged together with the processor of the pipeline data detection equipment, or it may be packaged separately from the processor of the pipeline data detection equipment; this application does not limit this.

[0049] The descriptions of the second, third, fourth, fifth, and sixth aspects of this application can be referenced to the detailed description of the first aspect. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a structural block diagram of a pipeline data detection system provided in an embodiment of the present invention;

[0052] Figure 2 A flowchart of a pipeline data detection method provided in an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of data acquisition points provided in an embodiment of the present invention;

[0054] Figure 4 A flowchart of another pipeline data detection method provided in an embodiment of the present invention;

[0055] Figure 5 A flowchart of another pipeline data detection method provided in an embodiment of the present invention;

[0056] Figure 6 A flowchart of another pipeline data detection method provided in an embodiment of the present invention;

[0057] Figure 7A flowchart of another pipeline data detection method provided in an embodiment of the present invention;

[0058] Figure 8 This is a schematic diagram of the identification model provided in an embodiment of the present invention;

[0059] Figure 9 This is a structural block diagram of a pipeline data detection device provided in an embodiment of the present invention;

[0060] Figure 10 This is a structural block diagram of a pipeline data detection device provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0063] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0064] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0065] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0066] Existing oil and gas pipeline data acquisition and monitoring systems do not perform reliability assessments on data collected continuously over a period of time at monitoring points (data acquisition points), only identifying missing data, data exceeding the measurement range, and data exceeding limits. Data completion using traditional mathematical methods (such as interpolation and moving averages) requires data before and after the missing data, thus failing to achieve real-time completion. When external environmental conditions fluctuate drastically, the time-series data may not conform to the premise of smooth change, leading to excessive errors between the completed data and the actual data.

[0067] Currently, oil and gas pipelines are equipped with a considerable number of monitoring devices, such as sensors for temperature, flow, and pressure, which can upload the collected data to the main control center. However, during the data acquisition and uploading process, issues such as sensor failure, communication breakdowns, packet loss, and unauthorized attacks may occur, preventing the main control center from obtaining accurate pipeline data. Existing solutions only identify missing, out-of-range, and limit-exceeding data, which cannot guarantee data integrity and accuracy.

[0068] In this scenario, the detection system faces the risk of erroneous commands due to incorrect data, while missing data can disrupt the normal operation of third-party simulation software for oil and gas pipelines. Current reliability assessments of pipeline detection information primarily rely on models built around physical properties, specifically using relationships between strongly correlated physical quantities such as pressure and density to determine if the data has been tampered with or transmitted incorrectly.

[0069] Specifically, physical models can detect the rationality of data based on the logical relationships and physical laws of these physical quantities, thereby identifying obvious anomalies. However, this method can only handle relatively simple tampering methods, i.e., tampering with individual data. For some more covert and complex tampering methods, traditional physical models are difficult to identify. For example, an attacker might systematically adjust all data in a data transmission to ensure consistent tampering while maintaining physical relationships. This operation usually makes the tampered data appear to conform to all physical laws, thus circumventing anomaly detection methods based on physical models.

[0070] To address this challenge, the reliability of data can be assessed by utilizing multiple datasets uploaded from the same detection point within a continuous timeframe. This method, through comparison and analysis of data from multiple time periods, can more effectively detect anomalies in continuous data, thus more reliably identifying potential tampering. However, this approach also has limitations. Because it relies on the accumulation of long-term data for sufficient comparison and analysis, a certain time delay occurs during the detection process. This delay may affect the system's real-time performance, hindering rapid response to data anomalies and leaving the system still at risk of generating erroneous instructions due to incorrect data.

[0071] To address this issue, this application provides a pipeline data inspection method. This method involves determining data anomalies in sampled data from multiple data collection points of the pipeline under inspection, thereby obtaining anomaly determination results and thus determining the integrity and / or reliability of the sampled data from these points. Specifically, the method determines whether the sampled data from multiple data collection points includes lost data, missing data, and garbled data, and whether there is erroneous data within the sampled data. Based on these anomaly determination results, the method determines the data inspection result for the pipeline under inspection, generates an error log, and issues an alarm.

[0072] In this way, by analyzing and judging the sampled data from multiple data collection points of the pipeline under inspection, it is possible to determine whether there is packet loss, missing data, garbled data, or erroneous data in the sampled data. Based on the judgment results, the data inspection result of the pipeline under inspection is determined, and an error log is generated and an alarm is issued to provide a prompt. This improves the accuracy of pipeline data inspection, accurately identifies abnormal data, effectively reduces the probability of pipeline failure, and ensures the safe operation of the pipeline.

[0073] This application provides a pipeline data detection system, such as... Figure 1 As shown, the pipeline data detection system 100 includes: a data acquisition module 101, an anomaly determination module 102, an anomaly alarm module 103, and a sensor 104.

[0074] The data acquisition module 101 is configured to acquire sampling data from multiple data acquisition points of the pipeline to be tested. The sampling data from each data acquisition point is time-series data, and the sampling data is used to characterize the operating characteristics of the pipeline to be tested.

[0075] Optionally, the data acquisition module 101 can be a data acquisition and uploading module, which is configured to acquire and upload data (such as flow rate, temperature, pressure, etc.) characterizing the operating features of the pipeline under test based on multiple sensors.

[0076] The anomaly detection module 102 is configured to perform data anomaly detection on the sampled data to obtain the anomaly detection result of the sampled data; the data anomaly detection includes: integrity detection and / or credibility detection; the integrity detection includes at least one of the following: packet loss detection, missing data detection and garbled data detection; the credibility detection is used to determine the erroneous data in the sampled data.

[0077] Optionally, the anomaly detection module 102 may specifically include: a data validity detection module and a data feasibility identification module. The data validity detection module is configured to determine the integrity and availability of uploaded data, mark and discard lost, incomplete, and garbled data. The data feasibility identification module is configured to input the uploaded data into a data confidence detector for multiple authentications, filter out erroneous data, mark it, and discard it.

[0078] The anomaly alarm module 103 is configured to determine the data detection result of the pipeline to be detected based on the anomaly judgment result, generate an error log and issue an alarm.

[0079] Optionally, the anomaly alarm module 103, also known as the anomaly status alarm module, is configured to generate an error log and issue an alarm for the occurrence of long-term continuous abnormal data.

[0080] The pipeline data detection method provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0081] The pipeline data detection method provided in this application embodiment is applied to Figure 1 The pipeline data detection system shown is as follows. Figure 2 As shown, the pipeline data detection method includes: S201-S203.

[0082] S201. Obtain sampling data from multiple data acquisition points of the pipeline to be tested.

[0083] The sampled data at each data acquisition point is time-series data, which is used to characterize the operational features of the pipeline under test.

[0084] Optionally, multiple data acquisition points are set on the pipeline to be inspected, and these data acquisition points can be located at the station location of the pipeline to be inspected. For each data acquisition point, various operational characteristic data of the pipeline can be collected through corresponding sensors, such as flow rate data, temperature data, pressure data, or density data.

[0085] It should be noted that various operational characteristic data of the pipeline under test are collected in real time at each data acquisition point, or the data can be collected once at preset intervals (30 seconds, one minute, five minutes, etc.). Therefore, the sampled data at each data acquisition point is time-series data.

[0086] In some embodiments of this application, the above-mentioned S201 can be implemented through the following steps:

[0087] S2011. Obtain sensor data collected by sensors at multiple data acquisition points on the pipeline to be inspected.

[0088] It is understandable that each data acquisition point is equipped with sensors for collecting various operational characteristic data of the pipeline under test, such as flow sensors, temperature sensors, pressure sensors, or density sensors.

[0089] In one possible implementation, sensors located upstream and downstream of the pipeline to be monitored collect different data characterizing the pipeline's operating status, perform preliminary processing, add time stamps to the data, and then upload the data to the main control center.

[0090] It is understandable that when sensors collect different data that characterize the pipeline's operating status, timestamps can also be added to the data using sensors.

[0091] In one possible implementation, after each data acquisition point collects sampling data through a sensor (sensing device), it can transmit (upload) the data to the main control center. The main control center then processes the sampling data from each data acquisition point to determine the data detection result of the pipeline to be inspected.

[0092] S2012. Perform data preprocessing on the sensor data to obtain the sampling data of each data acquisition point among multiple data acquisition points. Data preprocessing includes at least one of the following: adding timestamps, data normalization, and data cleaning.

[0093] In one possible implementation, the main control center can preprocess the sensor data from each of the multiple data acquisition points received to obtain the sampled data from each of the multiple data acquisition points.

[0094] Optionally, the methods for obtaining accurate time information through pipelines include, but are not limited to, using satellite time synchronization or obtaining clocks through network node communication, thereby determining the timestamp of the sampled data of each of the multiple data acquisition points.

[0095] For example, such as Figure 3As shown, taking five data acquisition points (stations) of the pipeline under inspection as an example, data acquisition points A, B, C, D, and E are located upstream and downstream of the pipeline under inspection, respectively, with their positional order being A, B, C, D, and E. The flow direction of the medium inside the pipeline is A, B, C, D, and E. The collected values ​​from each data acquisition point are sorted in chronological order at the upper-level node (master control center). Assume that the data (e.g., pressure data) at data acquisition point C at time t is abnormal (missing data) due to network attacks, sensor malfunctions, communication failures, etc. Figure 3 The black square in the diagram represents the abnormal data. The data collected by data collection point C within the time period T before the abnormal data is called time series data. Data collected by data collection points A, B, D, and E at time t and within the time period T before that time are sorted in chronological order to form four adjacent data sets.

[0096] It should be noted that the number of data collection points for the pipeline to be inspected in actual applications is set according to the actual situation.

[0097] S202. Perform data anomaly determination on the sampled data to obtain the anomaly determination result of the sampled data.

[0098] The data anomaly determination includes: integrity determination and / or credibility determination; integrity determination includes at least one of the following: packet loss determination, missing data determination, and garbled data determination; credibility determination is used to identify erroneous data in the sampled data.

[0099] Optionally, the main control center can determine the integrity and availability of uploaded data, mark lost data, incomplete data, and garbled data, and discard them.

[0100] Optionally, the main control center can input the uploaded data into the data confidence detector for multiple authentications, filter out erroneous data, mark it, and discard it.

[0101] S203. Determine the data detection results of the pipeline to be tested based on the anomaly judgment results, generate an error log and issue an alarm.

[0102] It is understandable that for data collection points that experience prolonged periods of continuous abnormal data, an error log and alert can be generated for that data collection point.

[0103] Optionally, for abnormal data, the abnormal data can be recorded in the error log of the corresponding data collection point, and an alarm message for that data collection point can be generated.

[0104] In some embodiments of this application, such as Figure 4As shown, when data anomaly detection includes integrity detection, and integrity detection includes packet loss detection, missing data detection, and garbled data detection, the above S202 can be implemented through the following steps:

[0105] S301. Regarding the sampled data of the first data acquisition point, if the main control center does not receive the data transmitted by the first data acquisition point at the data transmission time point, the anomaly judgment result is determined to be that there is packet loss in the sampled data of the first data acquisition point.

[0106] The first data acquisition point is any one of multiple data acquisition points, and the data transmission time point is the time when the data acquisition point transmits data to the main control center.

[0107] It is understandable that integrity determination is used to identify the integrity and availability of data, and to mark and discard lost data, incomplete data (missing data), and garbled data.

[0108] In one possible implementation, if the upper node (master control center) does not receive data transmitted from the lower node (data acquisition point, sensor) within a specified time, the upper node will record the corresponding data as lost data.

[0109] S302. When the data transmitted by the first data acquisition point received by the main control center fails the data bit verification, the anomaly determination result is that there is missing data in the sampled data of the first data acquisition point.

[0110] In one possible implementation, when the data received by the upper node (master control center) fails the data bit verification, the upper node (master control center) records the corresponding data as incomplete data (missing data).

[0111] S303. When the data transmitted by the first data acquisition point received by the main control center fails the data error correction verification, the anomaly determination result is that there is garbled data in the sampled data of the first data acquisition point.

[0112] In one possible implementation, when the data received by the upper node (master control center) fails the data verification and error correction check (e.g., cyclic redundancy check), the upper node records the corresponding data as garbled data.

[0113] In some embodiments of this application, when the data anomaly determination includes a credibility determination, the above-mentioned S202 can be implemented through the following steps:

[0114] S2021. For the sampled data of any one of the multiple data collection points, if the sampled data of any one data collection point exceeds the reference data threshold, the anomaly determination result is that there is erroneous data in the sampled data of any one data collection point.

[0115] In one possible implementation, the uploaded data (sampled data from data collection points) can be evaluated based on physical data to determine whether it conforms to the physical model (data correlation identification). Conforming to the physical model can be understood as follows: values ​​of temperature, pressure, and flow rate that are significantly higher or lower than normal values ​​are considered erroneous data.

[0116] In some embodiments of this application, the sampling data includes at least one of the following: flow rate data, temperature data, pressure data, and density data. The sampling data at each of the multiple data collection points includes data within a target time period. For example... Figure 5 As shown, when data anomaly detection includes credibility determination, the above S202 can be implemented through the following steps:

[0117] S401. Input the target data from the sampling data of the second data acquisition point and the sampling data of at least one third data acquisition point into the discrimination model to obtain the predicted data of the target data of the second data acquisition point at the target time.

[0118] Among them, at least one third data acquisition point is at least one data acquisition point that is adjacent to the second data acquisition point among multiple data acquisition points, the target data is any one of flow data, temperature data, pressure data, and density data, and the target time is the last time in the target time period.

[0119] Optionally, the uploaded data (sampled data from data collection points) can be input into the data confidence discriminator (i.e., the discrimination model) for multiple discriminations, filtering out erroneous data, marking it, and discarding it.

[0120] In one possible implementation, taking the pressure data from the sampling data of data acquisition point C as the target data, based on data correlation identification, the time series data (including flow data, temperature data, pressure data, and density data) of data acquisition point C and the adjacent data (pressure data) of one sampling point upstream and downstream (e.g., data acquisition points B and D) are input into the identification model, and the output result is the predicted data P of the pressure data of data acquisition point C at time t.

[0121] S402. When the difference between the predicted data and the target data at the second data acquisition point at the target time exceeds a preset threshold, the anomaly determination result is that the sampled data of at least one of the second data acquisition point and at least one third data acquisition point includes abnormal data.

[0122] Thus, if the error between the predicted data P and the actual value R of the pressure data in the sampled data uploaded by data collection point C exceeds the threshold, it is determined that at least one of the three data collection points B, C, and D has an anomaly in its sampled data at time t.

[0123] In some embodiments of this application, such as Figure 6 As shown, the step S402 above, "determining that the sampled data of at least one of the second data acquisition point and at least one third data acquisition point includes abnormal data," can be achieved through the following steps:

[0124] S501. For the fourth data acquisition point, determine the two data acquisition point groups adjacent to the fourth data acquisition point.

[0125] The fourth data acquisition point is any one of the second data acquisition point and at least one third data acquisition point, and the two data acquisition point groups include different data acquisition points.

[0126] In one possible implementation, if an anomaly is detected in the sampled data of at least one of the three data collection points B, C, and D at time t, then data correlation identification is performed twice more on each of the data collection points B, C, and D.

[0127] For example, taking data collection point C as an example, data collection points A, B, C and C, D, E can be regarded as a group of data collection points. The corresponding sampled data is input into the discrimination model. If the error between the output result of the discrimination model and the received actual value still exceeds the threshold, the sampled data of data collection point C at time t is determined to be abnormal.

[0128] It is understandable that when data acquisition point C is the fourth data acquisition point, the two data acquisition point groups adjacent to the fourth data acquisition point are: the first data acquisition point group includes data acquisition point A and data acquisition point B, and the second data acquisition point group includes data acquisition point D and data acquisition point E.

[0129] S502. For any one of the two data acquisition point groups, input the sampled data of the fourth data acquisition point and the target data in the sampled data of each data acquisition point included in the data acquisition point group into the discrimination model to obtain the predicted data of the target data of the fourth data acquisition point at the target time.

[0130] In one possible implementation, for the first group of data acquisition points, the sampled data from data acquisition point C, as well as the pressure data from the sampled data from data acquisition point A and the pressure data from the sampled data from data acquisition point B, can be input into the discrimination model to obtain the predicted pressure data of data acquisition point C at time t.

[0131] In one possible implementation, for the second group of data acquisition points, the pressure data from data acquisition point C, as well as the pressure data from data acquisition point D and data acquisition point E, can be input into the discrimination model to obtain the predicted pressure data of data acquisition point C at time t.

[0132] S503. When the difference between the predicted data determined based on each of the two data acquisition point groups and the target data of the fourth data acquisition point at the target time exceeds a preset threshold, the anomaly determination result is that the sampled data of the fourth data acquisition point includes abnormal data.

[0133] In one possible implementation, when the difference between the predicted data based on the first and second data acquisition point groups and the pressure data at data acquisition point C exceeds a preset threshold, it is determined that the sampled data at data acquisition point C includes abnormal data.

[0134] In some embodiments of this application, the discrimination model includes: a gated recurrent unit (GRU), a convolutional neural network (CNN), and an artificial neural network (ANN). The GRU is used to extract temporal features of the data, and the CNN is used to extract spatial features of the data. Figure 7 As shown, the above S401 can be implemented through the following steps:

[0135] S601. Input the sampled data from the second data acquisition point and the target data from the sampled data from at least one third data acquisition point into the gated loop unit (GRU) to obtain the timing characteristics.

[0136] S602. Input the target data from the sampling data of the second data acquisition point and the sampling data of at least one third data acquisition point into the convolutional neural network (CNN) to obtain spatial features.

[0137] S603. Input the temporal and spatial features into the artificial neural network (ANN) to obtain the predicted data of the target data at the second data acquisition point at the target time.

[0138] For example, such as Figure 8As shown, a feasible scheme for the discrimination model involves inputting temporal data and neighboring data into the model. The discrimination model consists of a gated recurrent unit (GRU), a convolutional neural network (CNN), and an artificial neural network (ANN). By inputting the sampled data (local data) from the second data acquisition point and the target data (neighboring data) from at least one third data acquisition point into the GRU and the CNN, temporal and spatial features can be obtained. Then, the temporal and spatial features are combined by the ANN, and the output value is the predicted target data of the second data acquisition point at the target time.

[0139] This application provides a pipeline data inspection method. By performing anomaly detection on sampled data from multiple data collection points of the pipeline under inspection, anomaly detection results are obtained, thereby determining the integrity and / or reliability of the sampled data from multiple data collection points of the pipeline under inspection. Specifically, by determining whether the sampled data from multiple data collection points of the pipeline under inspection includes lost data, missing data, and garbled data, and whether there is erroneous data in the sampled data, anomaly detection results are obtained. Based on the anomaly detection results, the data inspection results of the pipeline under inspection are determined, error logs are generated, and alarms are issued.

[0140] In this way, by analyzing and judging the sampled data from multiple data collection points of the pipeline under inspection, it is possible to determine whether there is packet loss, missing data, garbled data, or erroneous data in the sampled data. Based on the judgment results, the data inspection result of the pipeline under inspection is determined, and an error log is generated and an alarm is issued to provide a prompt. This improves the accuracy of pipeline data inspection, accurately identifies abnormal data, effectively reduces the probability of pipeline failure, and ensures the safe operation of the pipeline.

[0141] This application takes into account the continuity and correlation of oil and gas transmission within pipelines in both time and space. For each pipeline, data collection points are set up at various stations, and the data collected from these points exhibits strong correlation in both time and space. Utilizing this characteristic, artificial intelligence algorithms can be used to achieve high-precision, real-time data detection, ensuring the integrity and accuracy of the data.

[0142] Furthermore, this application proposes a spatiotemporal characteristic-based solution to address the challenges of missing and erroneous data in pipeline control systems. This method requires no additional equipment and relies solely on data collected by the pipeline system itself to detect abnormal data. Compared to completion schemes based on statistical data, this method offers higher accuracy, stronger real-time performance, and greater resistance to network attacks.

[0143] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] This application embodiment can divide the pipeline data detection device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0145] like Figure 9 The diagram shown is a structural schematic of a pipeline data detection device provided in an embodiment of this application. Figure 9 The pipeline data detection device shown includes: acquisition unit 901, judgment unit 902 and processing unit 903.

[0146] The acquisition unit 901 is used to acquire sampling data from multiple data acquisition points of the pipeline under test. The sampling data from each data acquisition point is time-series data, and the sampling data is used to characterize the operating characteristics of the pipeline under test.

[0147] The determination unit 902 is used to determine the data anomalies in the sampled data to obtain the anomaly determination result of the sampled data; the data anomaly determination includes: integrity determination and / or credibility determination; the integrity determination includes at least one of the following: packet loss determination, missing data determination, and garbled data determination; the credibility determination is used to determine the erroneous data in the sampled data;

[0148] Processing unit 903 is used to determine the data detection result of the pipeline to be detected based on the anomaly judgment result, generate error logs and issue alarms.

[0149] like Figure 10The diagram shown is a hardware structure schematic of a pipeline data detection device provided in an embodiment of this application. The pipeline data detection device includes: a processor 1001, a memory 1002, a communication interface 1003, and a bus 1004. The processor 1001, memory 1002, and communication interface 1003 can be connected via the bus 1004.

[0150] Processor 1001 is the control center of the pipeline data detection equipment. It can be a single processor or a collective term for multiple processing elements. For example, processor 1001 can be a general-purpose central processing unit (CPU) or other general-purpose processors. The general-purpose processor can be a microprocessor or any conventional processor.

[0151] As one embodiment, processor 1001 may include one or more CPUs, for example Figure 10 CPU 0 and CPU 1 are shown in the diagram.

[0152] The memory 1002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0153] In one possible implementation, the memory 1002 can exist independently of the processor 1001. The memory 1002 can be connected to the processor 1001 via a bus 1004 and is used to store instructions or program code. When the processor 1001 calls and executes the instructions or program code stored in the memory 1002, it can implement the pipeline data detection method provided in the following embodiments of this application.

[0154] In another possible implementation, the memory 1002 can also be integrated with the processor 1001.

[0155] The communication interface 1003 is used for connecting the pipeline data detection equipment to other devices via a communication network, which can be Ethernet, wireless access network, wireless local area network (WLAN), etc. The communication interface 1003 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0156] Bus 1004 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0157] It should be pointed out that, Figure 10 The structure shown does not constitute a limitation on pipeline data inspection equipment, except Figure 10 In addition to the components shown, the pipeline data detection device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

[0163] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of pipeline data detection, characterized by, include: The sampling data of multiple data acquisition points of the pipeline to be tested are acquired. The sampling data of each data acquisition point is time-series data. The sampling data is used to characterize the operating characteristics of the pipeline to be tested. The sampled data is subjected to data anomaly determination to obtain the anomaly determination result of the sampled data; The data anomaly determination includes: integrity determination and / or credibility determination; the integrity determination includes at least one of the following: packet loss determination, missing data determination, and garbled data determination; the credibility determination is used to identify erroneous data in the sampled data; Based on the anomaly determination result, the data detection result of the pipeline to be detected is determined, an error log is generated, and an alarm is issued.

2. The method of claim 1, wherein, When the data anomaly determination includes the integrity determination, and the integrity determination includes the packet loss data determination, the missing data determination, and the garbled data determination, the step of performing data anomaly determination on the sampled data to obtain the anomaly determination result of the sampled data includes: Regarding the sampled data from the first data acquisition point, when the main control center does not receive the data transmitted by the first data acquisition point at the data transmission time point, it is determined that the anomaly judgment result is that there is packet loss in the sampled data of the first data acquisition point. The first data acquisition point is any one of the plurality of data acquisition points, and the data transmission time point is the time point at which the data acquisition point transmits data to the main control center. When the data transmitted by the first data acquisition point received by the main control center fails the data bit verification, the anomaly determination result is that there is missing data in the sampled data of the first data acquisition point; When the data transmitted by the first data acquisition point received by the main control center fails the data error correction verification, the anomaly determination result is that there is garbled data in the sampled data of the first data acquisition point.

3. The method according to claim 1 or 2, characterized in that, The sampling data includes at least one of the following: flow rate data, temperature data, pressure data, and density data, and the sampling data of each of the plurality of data collection points includes data within the target time period; When the data anomaly determination includes the credibility determination, the step of performing data anomaly determination on the sampled data to obtain the anomaly determination result of the sampled data includes: The sampled data from the second data acquisition point and the target data from the sampled data from at least one third data acquisition point are input into the discrimination model to obtain the predicted data of the target data of the second data acquisition point at the target time. The at least one third data acquisition point is at least one data acquisition point adjacent to the second data acquisition point among the plurality of data acquisition points. The target data is any one of the flow data, the temperature data, the pressure data, and the density data. The target time is the last time in the target time period. When it is determined that the difference between the predicted data and the target data of the second data acquisition point at the target time exceeds a preset threshold, the anomaly determination result is determined to be that the sampled data of at least one of the second data acquisition point and the at least one third data acquisition point includes abnormal data.

4. The method of claim 3, wherein, The determination that the anomaly judgment result is that the sampled data of at least one of the second data acquisition point and the at least one third data acquisition point includes abnormal data, including: For the fourth data acquisition point, two data acquisition point groups adjacent to the fourth data acquisition point are determined. The fourth data acquisition point is any one of the second data acquisition point and the at least one third data acquisition point. The two data acquisition point groups include different data acquisition points. For any one of the two data acquisition point groups, the sampled data of the fourth data acquisition point and the target data in the sampled data of each data acquisition point included in the data acquisition point group are input into the discrimination model to obtain the predicted data of the target data of the fourth data acquisition point at the target time. When the difference between the predicted data determined based on each of the two data collection point groups and the target data of the fourth data collection point at the target time exceeds the preset threshold, the anomaly determination result is that the sampled data of the fourth data collection point includes abnormal data.

5. The method of claim 3, wherein, The discrimination model includes: a gated recurrent unit (GRU), a convolutional neural network (CNN), and an artificial neural network (ANN). The GRU is used to extract the temporal features of the data, and the CNN is used to extract the spatial features of the data. The step of inputting the sampled data from the second data acquisition point and the target data from the sampled data from at least one third data acquisition point into the discrimination model to obtain the predicted data of the target data from the second data acquisition point at the target time includes: The target data from the sampled data of the second data acquisition point and the sampled data of the at least one third data acquisition point are input into the gated loop unit (GRU) to obtain the time series characteristics. The target data from the sampled data of the second data acquisition point and the sampled data of the at least one third data acquisition point are input into the convolutional neural network (CNN) to obtain the spatial features; The temporal features and spatial features are input into the artificial neural network (ANN) to obtain the predicted data of the target data at the second data acquisition point at the target time.

6. The method of claim 1, wherein, When the data anomaly determination includes the credibility determination, the step of performing data anomaly determination on the sampled data to obtain the anomaly determination result of the sampled data includes: For the sampled data of any one of the plurality of data collection points, if the sampled data of any one of the data collection points exceeds the reference data threshold, the anomaly determination result is that there is erroneous data in the sampled data of any one of the data collection points.

7. The method of claim 1, wherein, The acquisition of sampling data from multiple data acquisition points of the pipeline to be tested includes: Acquire sensor data collected by sensors at the multiple data acquisition points of the pipeline to be inspected; The sensor data is preprocessed to obtain the sampling data of each of the plurality of data acquisition points. The data preprocessing includes at least one of the following: adding timestamps, data normalization, and data cleaning.

8. A pipeline data detection system characterized by, include: Data acquisition module, anomaly detection module, and anomaly alarm module; The data acquisition module is configured to acquire sampling data from multiple data acquisition points of the pipeline under test. The sampling data from each data acquisition point is time-series data, and the sampling data is used to characterize the operating characteristics of the pipeline under test. The anomaly detection module is configured to perform data anomaly detection on the sampled data to obtain the anomaly detection result of the sampled data; The data anomaly determination includes: integrity determination and / or credibility determination; the integrity determination includes at least one of the following: packet loss determination, missing data determination, and garbled data determination; the credibility determination is used to identify erroneous data in the sampled data; The anomaly alarm module is configured to determine the data detection result of the pipeline to be detected based on the anomaly determination result, generate an error log and issue an alarm.

9. A pipeline data detection device, characterized by, include: A processor and a memory; wherein the memory is used to store one or more programs, the one or more programs including computer-executable instructions, wherein when the pipeline data detection device is running, the processor executes the computer-executable instructions stored in the memory to cause the pipeline data detection device to perform the pipeline data detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the computer execution instructions stored in the computer-readable storage medium are executed by the processor of the pipeline data detection device, the pipeline data detection device is able to perform the pipeline data detection method as described in any one of claims 1-7.

Images