Pipeline leakage data acquisition method with self-calibration function
By classifying and dynamically adjusting the transmission strategy of pipeline leakage data, the problem of improper data processing in the existing technology is solved, and the timeliness, efficiency and accuracy of data is improved, ensuring the reliability of leakage monitoring.
Patent Information
- Application Number
- CN202510809411.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-17
- Publication Date
- 2025-08-15
AI Technical Summary
In the existing pipeline leakage data acquisition methods, the data lacks effective classification, resulting in improper real-time and non-real-time data processing, serious resource waste, and it is difficult to dynamically adjust the strategy during data transmission, affecting the real-time and completeness of the data, and the data processing method is single, which cannot effectively reduce redundancy and improve accuracy.
By obtaining the update frequency of pipeline leakage data, real-time and non-real-time data are processed separately; data delay is monitored in a fixed time period, and the transmission strategy is adjusted according to the delay mean and threshold comparison; duplicate and non-repetitive data are binary conversion and specific rules replacement; combined with historical data verification, we ensure the accuracy of the data format and content.
It realizes timely processing of real-time data, reduces resource waste, improves data transmission efficiency and accuracy, reduces the risk of misjudgment and misjudgment, and ensures the timeliness and reliability of leakage monitoring.
Smart Images
Figure CN120488160A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of data acquisition, in particular to a pipeline leakage data acquisition method with a self-calibration function. Background Art
[0002] In the field of pipeline transportation systems, pipeline leakage has always been a key factor affecting their safe and stable operation. The existing pipeline leakage data collection process faces many technical difficulties.
[0003] Traditional pipeline leakage data collection methods mainly face the following technical problems: First, data collection lacks effective classification and cannot distinguish between real-time and non-real-time data, resulting in the inability to process data with high real-time requirements in a timely manner, affecting the timeliness of leakage monitoring; non-real-time data is processed in the same way as real-time data, resulting in a waste of resources.
[0004] Secondly, during the data transmission process, it is difficult to dynamically adjust the transmission strategy according to the data delay. When the data delay is too high, the transmission cannot be optimized in time, which affects the real-time and integrity of the data and may cause the leakage information to not be discovered and processed in time.
[0005] Third, the data processing and replacement methods are single and cannot effectively reduce data redundancy and improve data transmission efficiency. In addition, the data verification and analysis are not comprehensive enough, and it is impossible to accurately judge whether the data is abnormal, which affects the accuracy and reliability of pipeline leakage monitoring. Summary of the Invention
[0006] In view of the shortcomings of the existing technology, the present invention provides a pipeline leakage data acquisition method with a self-calibration function, which solves the problem of difficulty in dynamically adjusting the transmission strategy according to the data delay during data transmission.
[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a pipeline leakage data acquisition method with a self-calibration function, the method specifically comprising the following steps: Obtain pipeline leakage data update frequency, and divide it into real-time and non-real-time data according to the frequency; Calculate the mean real-time data transmission delay and compare it with the threshold to generate a transmission adjustment analysis signal; Analyze the adjustment signal, calculate the matching ratio of the average delay to the unit delay within the time period, then compare it with the ratio threshold, determine the transmission standard, and generate data transmission information; Perform data transmission processing based on data transmission information, identify duplicate and non-duplicate data in the same-frequency data, perform binary conversion on the two, generate a replacement template, and replace it with the original data to generate combined data for transmission; Obtain the combined data after transmission and verify the data format and data content, and verify the data content in combination with historical data of the same period to generate a calibration processing signal or a normal acquisition signal.
[0008] As a further solution of the present invention, the specific manner of dividing the data into real-time and non-real-time data is as follows: The historical data and data update frequency corresponding to the pipeline leakage data are obtained and compared with the preset value. The pipeline leakage data corresponding to the data update frequency greater than the preset value is classified as real-time data, otherwise it is classified as non-real-time data.
[0009] As a further solution of the present invention, the specific method of generating the transmission adjustment analysis signal is: Taking time t as the period, the data delay corresponding to the time period is collected, and the mean data delay is calculated. At the same time, the mean data delay is compared with the delay threshold. If the mean data delay is greater than the delay threshold, it means that transmission adjustment is required based on the current data delay, and a transmission adjustment analysis signal is generated. Conversely, if the mean data delay is less than the delay threshold, it means that no transmission adjustment is required.
[0010] As a further solution of the present invention, the specific method of analyzing and adjusting the signal is: Calculate the mean of the sum of the maximum and minimum data delays within time period t, compare the mean with the unit delay value, calculate the difference and compare it with the threshold set by the operator, count the number of matches and calculate the proportion, and compare it with the proportion threshold; If the ratio exceeds the threshold, the transmission frequency is set based on the average value. If the ratio does not exceed the threshold, the data delay stabilization stage is found, and the transmission frequency is set accordingly to generate real-time data transmission information. Non-real-time data is processed according to the normal method to generate corresponding transmission information.
[0011] As a further solution of the present invention, the specific manner of performing data transmission processing according to the data transmission information is: Obtain the corresponding data transmitted at the same frequency and label it as a, where a=1, 2, ..., b, where b represents the number of data. At the same time, identify the duplicate data in data a and classify them into duplicate data and non-duplicate data, and then replace the two respectively.
[0012] As a further solution of the present invention, the specific method of replacing duplicate data and non-duplicate data is as follows: The specific method of replacing duplicate data is to convert the duplicate data into binary, calculate the sum of all binary numbers, generate a replacement value based on the sum, and replace the original duplicate data with the generated replacement value to obtain replacement data; The specific method of replacing non-repeated data is to perform binary conversion on the non-repeated data, group them according to the number of binary numbers, generate a replacement template, and replace the original non-repeated data with the replacement template to obtain replacement data.
[0013] As a further solution of the present invention, the specific method of generating the calibration processing signal or the normal acquisition signal is: Obtain the transmitted pipeline leakage data, perform data verification and analysis on it, obtain the pipeline leakage data, and judge its data format with the original data format. If the two are the same, it means that the transmission is normal and no processing is required. Otherwise, if the two are different, it means that the transmission is abnormal and its data format needs to be corrected; Then the data content of the pipeline leakage data is obtained, and the historical data of the same period is obtained, and the two are compared and analyzed.
[0014] As a further solution of the present invention, the specific method of comparing and analyzing the two is as follows: Collect new data and historical data from the same period to ensure that the same monitoring indicators are covered. Use time as the horizontal axis and the monitoring indicators as the vertical axis to draw a time series graph of the new data and the historical data from the same period, and compare the change patterns of the two. If the new data shows an obvious rise, fall, or abnormal fluctuation, generate a calibration processing signal; if the change patterns are similar, generate a normal acquisition signal.
[0015] The present invention provides a pipeline leakage data acquisition method with a self-calibration function. Compared with the prior art, it has the following advantages: This invention classifies pipeline leakage data into real-time and non-real-time data by measuring its update frequency and comparing it with preset values. For real-time data such as pressure, flow, and temperature, high-frequency monitoring and targeted processing are performed to ensure timely capture of key pipeline leakage information. For non-real-time data, appropriate processing methods are used, resources are rationally allocated, and data processing efficiency is improved, effectively enhancing the timeliness and accuracy of pipeline leakage monitoring.
[0016] This method monitors real-time data transmission delays at fixed intervals and flexibly adjusts transmission strategies based on the comparison of mean delays with thresholds. When delays are too high, the system analyzes the maximum and minimum data delays and sets transmission frequencies accordingly, ensuring data transmission stability and effectiveness, minimizing the impact of data delays on leakage monitoring, and improving data real-time performance and integrity.
[0017] This invention performs binary conversion and rule-based replacement on duplicate and non-duplicate data in transmitted data, effectively reducing data redundancy and improving data transmission efficiency. Furthermore, a checksum is added before data transmission, and the format and content are rigorously verified after transmission. Combined with comparative analysis of historical data from the same period, this ensures data accuracy and reliability, providing powerful data support for pipeline leakage monitoring and reducing the risk of misjudgments and missed detections due to data issues.
[0018] The present invention predefines the data format standard, compares the data formats after transmission one by one, and corrects abnormal formats in a timely manner; through various methods such as drawing time series graphs or setting thresholds, the data content is comprehensively verified in combination with historical data of the same period to accurately determine whether there are abnormalities in the data. Compared with the single data verification method of the existing technology, the reliability of the data is greatly improved, providing a solid guarantee for the accurate judgment and timely treatment of pipeline leakage. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a diagram of the steps of the present invention. DETAILED DESCRIPTION
[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0021] Example 1 See also Figure 1 The present application provides a pipeline leakage data acquisition method with a self-calibration function, which specifically includes the following steps: Step 1: Obtain the required pipeline leakage data, which includes pressure, flow, temperature, sound wave or vibration data, image data, and potential data, and classify them into real-time data and non-real-time data. The specific classification method is as follows: Obtain historical data corresponding to pipeline leakage data, and obtain the data update frequency corresponding to the pipeline leakage data, and compare it with the preset value. The specific value of the preset value is set by the operator. The pipeline leakage data corresponding to the data update frequency greater than the preset value is classified as real-time data, and the pipeline leakage data corresponding to the data update frequency less than the preset value is classified as non-real-time data.
[0022] For example, parameters such as pressure, flow, and temperature in a pipeline need to be obtained through continuous monitoring and high-frequency data collection, and they should be classified as real-time data accordingly.
[0023] Step 2: Monitor the data delay during the acquisition of real-time data (such as pipeline pressure, flow, temperature, etc.) using time t as a fixed period. In each time period t, record the delay time of each data transmission and use the formula Calculate the mean data delay , where d i represents the delay time of the i-th data transmission, and n is the number of data transmissions within time t; The calculated data delay mean The delay threshold D set by the operator th For comparison, if >D th , it indicates that the current data transmission delay is too high, which may affect the real-time performance of pipeline leakage monitoring and requires transmission adjustment. At this time, a transmission adjustment analysis signal is generated. If ≤D th , it means that the data transmission delay is within an acceptable range and no transmission adjustment is required; After the transmission adjustment analysis signal is generated, the data delay within the time period t is further analyzed. The maximum value d of the data delay within the period is obtained. max and the minimum value d min , through the formula Calculate the mean m of the sum of the two, and match the mean m with the unit delay value within the time period u set by the operator, that is, calculate the difference ,like Less than the matching threshold set by the operator , then the two are judged to match, otherwise they are judged to not match; Count the number of matches and calculate the proportion of the number of matches , where c is the number of matches, if p exceeds the threshold P set by the operator th , indicating that the mean m is well representative, and the transmission is performed based on the mean m, and the corresponding transmission frequency f is set according to the mean m. For example, if the mean m is small, the transmission frequency f can be appropriately increased, and if the mean m is large, the transmission frequency f can be appropriately reduced; If p does not exceed the threshold value P set by the operator th , the algorithm identifies the stable phase of data delay within time period t (i.e., the period during which the difference between data delays fluctuates within a small range). Transmission is based on the data delay during the stable phase and the corresponding transmission frequency f is set. The start and end times of the stable phase are also recorded to provide a reference for subsequent analysis. Assume that in a pipeline leakage monitoring system, the operator sets the time period t = 60 minutes and the delay threshold D th=5 seconds, unit delay value u=3 seconds, matching threshold =1 second, proportion threshold P th =70%, in a time period of t=60 minutes, a total of 120 transmissions were carried out, and the delay time of each data transmission was recorded. The average data delay was calculated. = 6 seconds, because Greater than D th = 5 seconds, so a transmission adjustment analysis signal is generated, and the data delay within this period is further analyzed to obtain the maximum value d max = 10 seconds, minimum value d min = 2 seconds, calculate the mean m = 6 seconds, the difference = 3 seconds, due to =3 seconds greater than = 1 second, so the two do not match. The number of matches is c = 30, and the proportion of calculations is p = 25%, because p < P th =70%, so the algorithm recognizes that the data delay is in a stable stage between the 20th and 40th minutes (for example, the delay time fluctuates between 4 and 5 seconds). Based on the data delay in this stable stage, the transmission frequency f is set. For example, the transmission frequency is adjusted from 2 times per minute to 1 time per minute to ensure the stability and effectiveness of data transmission.
[0024] Step 3: Perform transmission processing based on the obtained data transmission information, obtain corresponding data transmitted at the same frequency, and label them as a, where a=1, 2, ..., b, where b represents the number of data. At the same time, identify duplicate data in data a and classify them into duplicate data and non-duplicate data. Then, replace the obtained duplicate data and non-duplicate data respectively to obtain different replacement data. The specific method of replacing duplicate data is to obtain duplicate data, perform binary conversion on the duplicate data to obtain binary conversion data, obtain all binary numbers at the same time, calculate the sum of the binary values, then generate a replacement value based on the sum of the obtained binary values, and replace the obtained replacement value with the corresponding duplicate data; The specific method of replacing non-repeated data is to perform binary conversion on the non-repeated data, and obtain the binary number of the corresponding group number according to the number of binary numbers, where the group number is represented as the binary group number 01, generate a replacement template, and replace the obtained replacement template with the non-repeated data to obtain replacement data; The obtained replacement data are combined to obtain combined data, and the combined data are transmitted. The combination method here is to combine according to the position order of the original data. Similarly, the obtained non-real-time data is processed in the same way.
[0025] According to the data transmission information, the pipeline leakage data (such as pressure, flow, etc.) transmitted at the same frequency is obtained and numbered a1, a2, ..., a in the order of data reception. b , where b is the total number of data. Use hashing algorithms or sorting comparisons to quickly identify duplicate and non-duplicate data in a dataset. Identical data is classified as duplicate, and the rest as non-duplicate. For example, for the pressure data sequence [10, 10, 12, 10, 15], data 10 is identified as a duplicate, while data 12 and 15 are non-duplicate. The method for replacing duplicate data is to convert the value of each set of duplicate data into binary form. For example, the value 10 is converted to binary form as 1010. Then, all the binary numbers corresponding to the set of duplicate data are added together (if the number of bits is different, the high bit is filled with 0) to get the sum of the binary values. Based on the sum of binary values, a replacement value is generated through a modulo operation or a specific mapping rule. For example, if the sum of the binary values is 10101, it can be modulo 100 to obtain a replacement value of 21, and the generated replacement value is used to replace the original repeated data. If there are multiple 10s in the original data set, they are all replaced with 21; To replace non-repeating data, convert each non-repeating data into binary form. Based on the total number of binary digits, divide the binary digits into several groups (each group contains one or more binary digits), use "01" as the group identifier, and generate a replacement template. For example, the binary numbers 101, 110, and 111 can be divided into two groups, with the template [01, 101, 01, 110, 111]. The non-repeating data is replaced according to the template rules to generate new replacement data. Assume that a set of pipeline pressure data [15, 15, 18, 15, 20] is obtained and processed according to the above process: the data are labeled a1=15, a2=15, a3=18, a4=15, a5=20. After identification, 15 is repeated data, 18 and 20 are non-repeated data, 15 is converted to binary 1111, there are 3 15s in this group, the sum of the binary numbers is 1111+1111+1111=101101, 100 modulo 53 is obtained, all 15s in the data set are replaced with 53, 18 is converted to binary 10010, 20 is converted to binary 10100, the binary numbers are divided into two groups, and a replacement template [01, 10010, 01, 10100] is generated, and the template is used to replace the original non-repeated data; Combine the data in the original order to obtain a new data sequence [53, 53, 01, 10010, 53, 01, 10100], add the check code and transmit it.
[0026] Step 4. Obtain the pipeline leakage data after transmission, and perform data verification and analysis on it. Clarify the original format standard of the pipeline leakage data in advance, including the data type (such as floating point type for pressure data and integer type for flow data), data length (such as temperature data retains two decimal places), encoding method (UTF-8, etc.), timestamp format (YYYY-MM-DDHH:MM:SS) and unit (pressure is MPa, flow is m³ / h, etc.). Obtain the pipeline leakage data after transmission and compare its data format with the original format standard one by one. If all format elements are the same as the original format, it is determined that the transmission is normal and no processing is performed; if there is any format inconsistency, such as non-numeric characters in pressure data, incorrect timestamp format, etc., it is immediately marked as a transmission anomaly. For data with transmission anomalies, start the format correction process and convert, complete or re-encode the erroneous part according to the original format standard. For example, adjust the erroneous timestamp format to the standard format to ensure that the data format meets the requirements; Collect the transmitted pipeline leakage data and corresponding data from the same time period (historical period), covering the same monitoring indicators such as pressure, flow, temperature, sound waves, or vibration data. Check the data integrity and remove data records with excessive missing values or obvious errors. Standardize the data format and convert units to ensure that all data has the same timestamp or time interval to prepare for subsequent comparative analysis. Draw a time series graph, displaying the new data and historical data for the same period on the same chart with time as the horizontal axis and each monitoring indicator as the vertical axis. Observe the trends of the new data curve and the historical data curve for the same period to determine whether the new data has a similar change pattern to the historical data for the same period. If the new data curve shows a significant rise, fall, or abnormal fluctuation within a certain period of time, and the difference is significant from the historical data for the same period, the data is considered abnormal and a calibration processing signal is generated. If the difference is small, a normal acquisition signal is generated, indicating that the data content is reliable.
[0027] Example 2 As the second embodiment of the present invention, it is implemented on the basis of the first embodiment, and differs from the first embodiment in the following aspects: The method of comparing the pipeline leakage data with the historical data of the same period in step 4 is different. In this embodiment, a corresponding comparison threshold is set to determine whether the collected pipeline leakage data is abnormal. The specific analysis method is as follows: Thresholds are set to identify outliers in new data. The range of normal data can be determined based on the statistical characteristics of historical data over the same period, such as the mean plus or minus a certain number of standard deviations. Data points outside this range are considered outliers. These outliers are then flagged and analyzed in detail to determine whether they are caused by special circumstances such as measurement error, equipment failure, or actual pipeline leakage.
[0028] Example 3 As the third embodiment of the present invention, the focus is on combining the implementation processes of the first and second embodiments.
[0029] Some of the data in the above formulas are calculated based on their numerical values and are not substituted into parameter units for calculation. At the same time, the contents not described in detail in this specification belong to the existing technology known to those skilled in the art.
[0030] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.
Claims
1. A pipeline leakage data acquisition method with self-calibration function, characterized in that: The method specifically comprises the following steps: Obtain pipeline leakage data update frequency, and divide it into real-time and non-real-time data according to the frequency; Calculate the mean real-time data transmission delay and compare it with the threshold to generate a transmission adjustment analysis signal; Analyze the adjustment signal, calculate the matching ratio of the average delay to the unit delay within the time period, then compare it with the ratio threshold, determine the transmission standard, and generate data transmission information; Perform data transmission processing based on data transmission information, identify duplicate and non-duplicate data in the same-frequency data, perform binary conversion on the two, generate a replacement template, and replace it with the original data to generate combined data for transmission; Obtain the combined data after transmission and verify the data format and data content, and verify the data content in combination with historical data of the same period to generate a calibration processing signal or a normal acquisition signal.
2. The pipeline leakage data acquisition method with self-calibration function according to claim 1 is characterized in that: The specific method of dividing the data into real-time and non-real-time data is as follows: The historical data and data update frequency corresponding to the pipeline leakage data are obtained and compared with the preset value. The pipeline leakage data corresponding to the data update frequency greater than the preset value is classified as real-time data, otherwise it is classified as non-real-time data.
3. The pipeline leakage data acquisition method with self-calibration function according to claim 1 is characterized in that: The specific method of generating the transmission adjustment analysis signal is: Taking time t as the period, the data delay corresponding to the time period is collected, and the mean data delay is calculated. At the same time, the mean data delay is compared with the delay threshold. If the mean data delay is greater than the delay threshold, it means that transmission adjustment is required based on the current data delay, and a transmission adjustment analysis signal is generated. Conversely, if the mean data delay is less than the delay threshold, it means that no transmission adjustment is required.
4. The pipeline leakage data acquisition method with self-calibration function according to claim 1 is characterized in that: The specific method of analyzing and adjusting the signal is as follows: Calculate the mean of the sum of the maximum and minimum data delays within time period t, compare the mean with the unit delay value, calculate the difference and compare it with the threshold set by the operator, count the number of matches and calculate the proportion, and compare it with the proportion threshold; If the ratio exceeds the threshold, the transmission frequency is set based on the average value. If the ratio does not exceed the threshold, the data delay stabilization stage is found, and the transmission frequency is set accordingly to generate real-time data transmission information. Non-real-time data is processed according to the normal method to generate corresponding transmission information.
5. The pipeline leakage data acquisition method with self-calibration function according to claim 1 is characterized in that: The specific method of performing data transmission processing according to the data transmission information is: Obtain the corresponding data transmitted at the same frequency and label it as a, where a=1, 2, ..., b, where b represents the number of data. At the same time, identify the duplicate data in data a and classify them into duplicate data and non-duplicate data, and then replace the two respectively.
6. The pipeline leakage data acquisition method with self-calibration function according to claim 5, characterized in that: The specific method of replacing duplicate data and non-duplicate data is as follows: The specific method of replacing duplicate data is to convert the duplicate data into binary, calculate the sum of all binary numbers, generate a replacement value based on the sum, and replace the original duplicate data with the generated replacement value to obtain replacement data; The specific method of replacing non-repeated data is to perform binary conversion on the non-repeated data, group them according to the number of binary numbers, generate a replacement template, and replace the original non-repeated data with the replacement template to obtain replacement data.
7. The pipeline leakage data acquisition method with self-calibration function according to claim 1 is characterized in that: The specific method of generating the calibration processing signal or the normal acquisition signal is: Obtain the transmitted pipeline leakage data, perform data verification and analysis on it, obtain the pipeline leakage data, and judge its data format with the original data format. If the two are the same, it means that the transmission is normal and no processing is required. Otherwise, if the two are different, it means that the transmission is abnormal and its data format needs to be corrected; Then the data content of the pipeline leakage data is obtained, and the historical data of the same period is obtained, and the two are compared and analyzed.
8. The pipeline leakage data acquisition method with self-calibration function according to claim 7, characterized in that: The specific method for comparing the two is as follows: Collect new data and historical data from the same period to ensure that they cover the same monitoring indicators. Draw a time series graph of the new data and the historical data from the same period, with time as the horizontal axis and the monitoring indicators as the vertical axis, and compare the change patterns of the two. If the new data shows a significant increase, decrease, or abnormal fluctuation, generate a calibration processing signal; If the change patterns are similar, a normal acquisition signal is generated.