SCADA (supervisory control and data acquisition) data denoising method and system for oil-gas pipe network and related equipment

Abstract

The invention relates to the technical field of oil and gas pipe networks, and discloses an SCADA data denoising method and system for an oil and gas pipe network and related equipment, and the method comprises the steps: obtaining historical SCADA data of the oil and gas pipe network; performing model training on the deep learning model for training wavelet transform based on historical SCADA data to obtain an optimal wavelet transform parameter; and carrying out wavelet transform denoising processing on SCADA data to be denoised based on the optimal wavelet transform parameter. The SCADA data is subjected to joint denoising through deep learning and wavelet transformation, the advantages that the deep learning model can automatically learn and optimize data and improve the wavelet transformation efficiency and accuracy can be fully utilized, local features and detail information in the signals can be effectively extracted through wavelet transformation, and the accuracy of the signals is improved. And noise and interference in the SCADA data are effectively removed.

Description

Technical Field

[0001] The present invention relates to the technical field of oil and gas pipeline networks, and in particular to a SCADA data denoising method, system and related equipment for oil and gas pipeline networks. Background Art

[0002] Data acquisition and processing is an important part of the normal operation of the oil and gas pipeline network. It is necessary to install corresponding on-site signal acquisition and processing systems on the pipeline to realize real-time acquisition and rapid processing of pipeline data, so as to determine that there are no abnormal situations such as leakage in the pipeline and ensure the normal operation of the pipeline network. Usually, a workstation is set up at the pipeline operation site, and an industrial computer is installed for data processing and transmission. Domestic oil fields are distributed over a wide area, and the transmission routes are relatively long. There are many points that need to be monitored and collected. Long-distance pipeline monitoring requires monitoring and scheduling of multiple collection points. Setting up a workstation at each site is too costly.

[0003] In response to the above problems, the Supervisory Control And Data Acquisition (SCADA) with information collection and transmission functions is applied in the monitoring system of pipeline operation. This application can realize remote monitoring and management control of pipelines without setting up sites, and is suitable for monitoring objects with many collection points, long communication lines, and wide distribution. SCADA needs to collect a large amount of data in real time, including pipeline pressure, temperature, flow and other parameters, as well as environmental factors such as wind speed and temperature. These data are not only huge in quantity, but also need to be collected and processed continuously; at the same time, due to various emergencies that may exist during the operation of oil and gas pipeline networks, such as pipeline leakage and pressure fluctuations, the transmission of monitoring data must be real-time so that these situations can be discovered and handled in time; therefore, it is necessary to clean, pre-process and analyze SCADA data to extract useful information and knowledge in order to better guide the operation and maintenance of pipelines.

[0004] There are many existing denoising methods, among which the faster and more effective method is to use the wavelet transform method. However, wavelet transform denoising has certain disadvantages, such as the need to select a suitable mother wavelet function and the number of wavelet decomposition layers. If the selection is inappropriate, it may lead to poor denoising effect or loss of too many useful signals. When processing data with complex structures or high-frequency signals, some difficulties may be encountered. Therefore, there is an urgent need for a denoising method suitable for SCADA data for oil and gas pipeline networks that can process complex and large data and is fast and effective. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a SCADA data denoising method, system and related equipment for oil and gas pipeline network in view of the problems existing in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a SCADA data denoising method for an oil and gas pipeline network, comprising: Obtain historical SCADA data for oil and gas pipeline networks; Based on historical SCADA data, the deep learning model of the wavelet transform to be trained is trained to obtain the optimal wavelet transform parameters; The SCADA data to be denoised is subjected to wavelet transform denoising based on the optimal wavelet transform parameters.

[0007] In a second aspect, an embodiment of the present invention provides a SCADA data denoising system for an oil and gas pipeline network, comprising: Data acquisition module, used to obtain historical SCADA data of oil and gas pipeline networks; A wavelet transform parameter determination module is used to train the deep learning model of the wavelet transform to be trained based on historical SCADA data to obtain the optimal wavelet transform parameters; The wavelet denoising module is used to perform wavelet transform denoising on the SCADA data to be denoised based on the optimal wavelet transform parameters.

[0008] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the SCADA data denoising method for oil and gas pipeline networks provided by the above technical solution.

[0009] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the SCADA data denoising method for oil and gas pipeline networks provided by the above technical solution is implemented.

[0010] In a fifth aspect, an embodiment of the present invention further provides a computer program product or a computer program, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the SCADA data denoising method for oil and gas pipeline networks provided in the above-mentioned various optional embodiments.

[0011] The beneficial effects of the present invention are as follows: by jointly denoising SCADA data through deep learning and wavelet transform, not only can the advantages of deep learning models that can automatically learn and optimize data and improve the efficiency and accuracy of wavelet transform be fully utilized, but also local features and detail information in the signal can be effectively extracted through wavelet transform, effectively removing noise and interference in SCADA data.

[0012] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A flow chart of a SCADA data denoising method for oil and gas pipeline networks shown as an exemplary embodiment of the present application; Figure 2 for Figure 1 A flowchart of step S102 in the illustrated embodiment in an exemplary embodiment; Figure 3 for Figure 2 A flowchart of step S203 in the illustrated embodiment in an exemplary embodiment; Figure 4 for Figure 3 A flowchart of step S302 in the illustrated embodiment in an exemplary embodiment; Figure 5 for Figure 1 A flowchart of step S103 in the illustrated embodiment in an exemplary embodiment; Figure 6 A block diagram of a SCADA data denoising system for an oil and gas pipeline network is shown as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0014] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0015] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0016] First see Figure 1 , Figure 1 This is a flowchart of a SCADA data denoising method for oil and gas pipeline networks shown in an exemplary embodiment of the present application. The method can be specifically executed by a server, which can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc., and is not limited here.

[0017] like Figure 1 As shown, in an exemplary embodiment, the SCADA data denoising method for oil and gas pipeline network may include steps S101 to S103, which are described in detail as follows: S101, obtaining historical SCADA data of the oil and gas pipeline network.

[0018] The oil and gas pipeline SCADA system collects data from various nodes of the oil and gas pipeline through sensors, instruments and other devices, including temperature, pressure, flow rate, etc. These data are collected in the form of time series to ensure the comprehensiveness and accuracy of data collection.

[0019] In an embodiment of the present invention, historical SCADA data may include basic node information of the oil and gas pipeline, node monitoring signals, and abnormal signals in the node monitoring signals, etc. The basic node information may include node number and node location information, etc. The node monitoring signal refers to the actual signal including temperature, pressure, flow rate, etc. collected at the oil and gas pipeline node, and the abnormal signal refers to the signal that the actual signal is judged to be abnormal or problematic in the oil and gas pipeline network after processing.

[0020] S102, based on historical SCADA data, a deep learning model of the wavelet transform to be trained is trained to obtain optimal wavelet transform parameters.

[0021] Among them, the quasi-trained wavelet transform is a limitation of the deep learning model, specifically referring to: using the input data as the training parameters of the wavelet transform deep learning model to train the deep learning model.

[0022] The embodiment of the present invention uses historical SCADA data to train the deep learning model of the wavelet transform to be trained, and automatically obtains the optimal wavelet transform parameters, which effectively solves the problem in the prior art that the mother wavelet function and the improper number of wavelet decomposition layers are manually selected, which may lead to poor denoising effect or excessive loss of useful signals. The method of the embodiment of the present invention can process data with complex structures or high-frequency signals.

[0023] S103, performing wavelet transform denoising processing on the SCADA data to be denoised based on the optimal wavelet transform parameters.

[0024] In the embodiment of the present invention, the SCADA data is jointly denoised by deep learning and wavelet transform, which can not only fully utilize the advantages of deep learning model that can automatically learn and optimize data and improve the efficiency and accuracy of wavelet transform, but also effectively extract local features and detail information in the signal through wavelet transform, and effectively remove noise and interference in SCADA data. In addition, the method of the embodiment of the present invention can also effectively solve some problems that are difficult to handle using one of the technologies alone, such as edge effect, wavelet basis selection, etc.

[0025] See also Figure 2 , Figure 2 for Figure 1 The step S102 in the illustrated embodiment is a flow chart in an exemplary embodiment. Figure 2 As shown, step S102 may specifically include steps S201 to S203. The optimal wavelet transform parameters are obtained through the above steps, which are described in detail as follows: S201, constructing a sample set consisting of node monitoring signals and abnormal signals; S202, using the node monitoring signals and abnormal signals in the sample set as input data and output data of the deep learning model respectively; S203, optimizing the wavelet function, decomposition level and threshold in the wavelet transform based on the deep learning model to obtain optimal wavelet transform parameters.

[0026] The embodiment of the present invention uses a large amount of labeled data for model training, and optimizes the parameters and structure of the model through the back propagation algorithm, so that the model can automatically learn and extract the characteristics and rules of the data. This embodiment does not limit the selection of the deep learning model.

[0027] The embodiments of the present invention make full use of the advantage that the deep learning model can automatically learn and optimize data, and automatically determine the optimal wavelet transform parameters through the deep learning model, thereby improving the efficiency and accuracy of the wavelet transform.

[0028] See also Figure 3 , Figure 3 for Figure 2 The step S203 in the embodiment shown is a flow chart in an exemplary embodiment. Figure 3 As shown, step S203 may specifically include steps S301 to S303, and the optimal wavelet transform parameters are obtained through the above steps, which are described in detail as follows: S301, inputting input data into the deep learning model.

[0029] In the embodiment of the present invention, the input data is the actual signal collected in S1, and the output data is the detection signal corresponding to the actual signal and judged as a problem or abnormality in the oil and gas pipeline network after processing. The output data should be a signal that has been manually confirmed and has a certain meaning.

[0030] S302, continuously iterating the wavelet function, decomposition level and threshold in the deep learning model through the back propagation algorithm to obtain the model error of the deep learning model.

[0031] The training of deep learning models requires the use of training data. The training data can be synthetic data or actual data (i.e., relevant data collected by the SCADA system). The actual signal used as the input data contains noise or a signal with unclear information. Since the wavelet transform has the function of denoising and highlighting the information in the data, the output signal is a denoised or highlighted signal. The error data generated by comparing the wavelet transform parameters after deep learning model training with the output data can be used to determine whether the model is well trained.

[0032] Since wavelet transform has the functions of denoising and highlighting information in data, both of which are related to the wavelet function, decomposition level and threshold of wavelet transform, this embodiment automatically learns the above three parameters through a deep learning model to obtain corresponding data.

[0033] S303, when the model error meets the preset iteration requirement, the iteration is stopped, and the wavelet function, decomposition level and threshold corresponding to the minimum value of the model error are obtained and used as the optimal wavelet transform parameters.

[0034] Specifically, during the training of the deep learning model, a stopping condition needs to be set. When the iteration reaches the maximum number of iterations or the error data reaches the iteration threshold, the iteration can be stopped.

[0035] The maximum number of iterations and the iteration threshold are parameters that need to be set during the iteration process. The maximum number of iterations specifies the maximum number of iterations, while the iteration condition threshold means stopping the iteration when a certain accuracy is reached. Therefore, when choosing these two parameters, you need to select them according to the specific situation.

[0036] When the maximum number of iterations is large, you can consider selecting the iteration condition threshold. The iteration condition threshold is usually a small value. The iteration is stopped when this value is reached. Usually, the iteration condition threshold should be as small as possible to ensure that the iteration obtains more accurate results. But at the same time, the iteration condition threshold cannot be too small, otherwise it will cause the iteration process to be too complicated and consume a lot of time and computing resources.

[0037] The iteration condition threshold can be adjusted according to actual needs. If more accurate results are required, the iteration condition threshold can be reduced; if the iteration speed is too slow, it can be increased appropriately.

[0038] In short, when choosing the maximum number of iterations and the iteration condition threshold, it is necessary to comprehensively consider the accuracy and efficiency of the algorithm. Selecting appropriate parameters can make the algorithm obtain more accurate and faster results.

[0039] See also Figure 4 , Figure 4 for Figure 3 The step S302 in the embodiment shown is a flow chart in an exemplary embodiment. Figure 4 As shown, step S302 may specifically include steps S401 to S402. The model error of the deep learning model is obtained through the above steps, which is described in detail as follows: S401, using the wavelet function as a transfer function of the deep learning model and linking it with the connection weights and the number of nodes in the deep learning model.

[0040] In an embodiment of the present invention, a wavelet function is used as a transfer function of a deep learning model, the input data is multiplied by the connection weight and a wavelet transform is applied to calculate the output of each layer, and the calculation error between the output of the deep learning model and the target output is compared; the calculation error is used to update the connection weight, the gradient or derivative of the wavelet function is calculated, and the number of nodes is adjusted by iterative training.

[0041] S402, denoising is performed by continuously adjusting the decomposition level and threshold, and the wavelet function, decomposition level and threshold are continuously iterated to obtain the model error of the deep learning model.

[0042] In an embodiment of the present invention, while the deep learning model is being trained, the wavelet function, decomposition level and threshold are continuously iterated and optimized to reduce the training error and ultimately obtain the minimum error. Compared with the existing manual selection, the present invention automatically learns the above three parameters through deep learning to improve the efficiency and accuracy of wavelet transform.

[0043] See also Figure 5 , Figure 5 for Figure 1 The step S103 in the embodiment shown is a flow chart in an exemplary embodiment. Figure 5 As shown, step S103 may specifically include steps S501 to S502, and the SCADA data to be denoised is subjected to wavelet transform denoising through the above steps, which is described in detail as follows: S501, performing wavelet transform on the SCADA data to be denoised based on the optimal wavelet transform parameters to obtain wavelet coefficients.

[0044] The embodiment of the present invention obtains wavelet coefficients through wavelet transformation, can further extract local features and detail information in the signal, and performs threshold processing on the wavelet coefficients to further remove noise and interference.

[0045] S502, reconstructing the SCADA data based on the wavelet coefficients to obtain the SCADA data after joint denoising.

[0046] The embodiment of the present invention reconstructs the SCADA data based on the wavelet coefficients after wavelet transformation to obtain the SCADA data after joint denoising. Some post-processing, such as edge smoothing and normalization, can be performed to further optimize the denoising effect.

[0047] like Figure 6 As shown, in an exemplary embodiment, the SCADA data denoising system for oil and gas pipeline network may include a data acquisition module, a wavelet transform parameter determination module and a wavelet denoising module, which are described in detail as follows: Data acquisition module, used to obtain historical SCADA data of oil and gas pipeline networks; A wavelet transform parameter determination module is used to train the deep learning model of the wavelet transform to be trained based on historical SCADA data to obtain the optimal wavelet transform parameters; The wavelet denoising module is used to perform wavelet transform denoising on the SCADA data to be denoised based on the optimal wavelet transform parameters.

[0048] An embodiment of the present invention further provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the SCADA data denoising method for an oil and gas pipeline network as provided in the above embodiment.

[0049] An embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the SCADA data denoising method for oil and gas pipeline networks provided in the above embodiment is implemented.

[0050] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the SCADA data denoising method for oil and gas pipeline network provided in the above various optional embodiments.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0052] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0053] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0054] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0055] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A SCADA data denoising method for oil and gas pipeline network, characterized in that: include: Obtain historical SCADA data for oil and gas pipeline networks; Based on the historical SCADA data, a deep learning model for wavelet transform to be trained is trained to obtain optimal wavelet transform parameters; The SCADA data to be denoised is subjected to wavelet transform denoising processing based on the optimal wavelet transform parameters.

2. The method according to claim 1, characterized in that The historical SCADA data includes: Basic information of nodes of oil and gas pipelines, node monitoring signals and abnormal signals in the node monitoring signals.

3. The method according to claim 2, characterized in that Based on the historical SCADA data, a deep learning model to be trained for wavelet transform is trained to obtain optimal wavelet transform parameters, including: Constructing a sample set consisting of the node monitoring signal and the abnormal signal; Using the node monitoring signal and the abnormal signal in the sample set as the input data and output data of the deep learning model respectively; Based on the deep learning model, the wavelet function, decomposition level and threshold in the wavelet transform are optimized to obtain the optimal wavelet transform parameters.

4. The method according to claim 3, characterized in that The step of optimizing the wavelet function, decomposition level and threshold in the wavelet transform based on the deep learning model to obtain the optimal wavelet transform parameters includes: Inputting the input data into the deep learning model; The wavelet function, decomposition level and threshold in the deep learning model are continuously iterated through a back propagation algorithm to obtain a model error of the deep learning model; When the model error meets the preset iteration requirement, the iteration is stopped, and the wavelet function, decomposition level and threshold corresponding to the minimum value of the model error are obtained and used as the optimal wavelet transform parameters.

5. The method according to claim 4, characterized in that The step of continuously iterating the wavelet function, decomposition level and threshold in the deep learning model through a back propagation algorithm to obtain a model error of the deep learning model includes: The wavelet function is used as a transfer function of the deep learning model and is associated with the connection weights and the number of nodes in the deep learning model; The denoising process is performed by continuously adjusting the decomposition level and the threshold, and the wavelet function, the decomposition level and the threshold are continuously iterated to obtain the model error of the deep learning model.

6. The method according to claim 5, characterized in that The method of using the wavelet function as a transfer function of a deep learning model and associating the wavelet function with a connection weight and a number of nodes in the deep learning model comprises: Using the wavelet function as a transfer function of a deep learning model, multiplying the input data by the connection weights and applying wavelet transform to calculate the output of each layer, and comparing the calculation error between the output of the deep learning model and the target output; The connection weights are updated using the calculation error, the gradient or derivative of the wavelet function is calculated, and the number of nodes is adjusted through iterative training.

7. The method according to any one of claims 1 to 6, characterized in that: The denoising process of performing wavelet transform on the SCADA data to be denoised based on the optimal wavelet transform parameters includes: Based on the optimal wavelet transform parameters, the SCADA data to be denoised is subjected to wavelet transform to obtain wavelet coefficients; The SCADA data is signal reconstructed based on the wavelet coefficients to obtain the SCADA data after joint denoising.

8. A SCADA data denoising system for oil and gas pipeline network, characterized in that: include: Data acquisition module, used to obtain historical SCADA data of oil and gas pipeline networks; A wavelet transform parameter determination module is used to perform model training on a deep learning model to be trained for wavelet transform based on the historical SCADA data to obtain optimal wavelet transform parameters; The wavelet denoising module is used for performing wavelet transform denoising processing on the SCADA data to be denoised based on the optimal wavelet transform parameters.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a computer, the computer is caused to execute the SCADA data denoising method for oil and gas pipeline network according to any one of claims 1 to 7.

10. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the SCADA data denoising method for oil and gas pipeline networks as described in any one of claims 1 to 7 is implemented.

Images