A pipeline safety early warning method and system based on big data

Abstract

The present invention relates to the technical field of pipeline safety monitoring, and in particular to a pipeline safety early warning method and system based on big data. The method comprises the steps of: processing each data point in the flow data sequence of the pipeline by an isolation forest algorithm, obtaining multiple isolated trees and the path length of each data point in the isolated tree; determining the preference of the isolated tree, the preference is positively correlated with the standard deviation of the path length of each data point in the isolated tree; obtaining the historical data segment of the data point in the flow data sequence, and calculating the consistency of the data point in the isolated tree; weighting the path length of the data point by the product of the consistency of the data point in the isolated tree and the preference of the isolated tree, processing the weighted average path length of the weighted data point by the isolation forest algorithm, obtaining the abnormal score of the data point, and obtaining the pipeline safety early warning result according to the comparison result of the abnormal score and the preset threshold, thereby effectively improving the accuracy of the pipeline safety early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline safety monitoring, and in particular to a pipeline safety early warning method and system based on big data. Background Art

[0002] Natural gas is one of the main energy sources in people's lives, and the widespread use of natural gas has made the safety issues of gas pipeline systems increasingly prominent. Gas in pipelines is a toxic, flammable and explosive medium. If gas leaks, it will directly lead to unsafe accidents. Especially when the pipeline has been buried for a long time, the aging, corrosion and looseness of the pipeline materials may lead to gas leakage, thus causing serious safety hazards.

[0003] The existing patent application document with publication number CN114547822A discloses a pipeline safety early warning method, device and medium. The application first obtains the early warning original data of each section of the pipeline; then, the attribute data and spatial data of each section of the pipeline are obtained according to the early warning original data of each section of the pipeline; then the pipeline safety early warning model is called to obtain the predicted remaining life of each section of the pipeline according to the attribute data of each section of the pipeline; finally, the remaining life of each section of the pipeline is compared with a preset threshold; if the remaining life of the pipeline is less than the preset threshold, an alarm is issued.

[0004] The above-mentioned prior art provides safety warning by obtaining the remaining life of the pipeline. Although it can reduce safety hazards to a certain extent, in the gas pipeline system, gas leakage may have occurred due to environmental influences, but the pipeline life detection cannot capture this change in time, and there are still major safety hazards.

[0005] Based on this, how to accurately implement pipeline safety early warning is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0006] In order to solve the technical problem of how to accurately implement pipeline safety early warning, the present invention provides a pipeline safety early warning method and system based on big data.

[0007] In the first aspect, the present invention provides a pipeline safety early warning method based on big data, which adopts the following technical solution:

[0008] A pipeline safety early warning method based on big data, comprising the steps of:

[0009] Process each data point in the flow data sequence of the pipeline by using the isolation forest algorithm to obtain multiple isolated trees and the path length of each data point in the isolated tree; determine the preference of the isolated tree, which is positively correlated with the standard deviation of the path length of each data point in the isolated tree; obtain the historical data segment of the data point in the flow data sequence, and calculate the consistency of the data point in the isolated tree: ; In the formula, represents the consistency of the jth data point in the i-th isolated tree, Represents the total number of data points in the parent node of the node where the jth data point is located in the i-th isolated tree, represents the flow value of the jth data point, Indicates the number of data points in the historical data segment, Indicates the number of historical data segments The flow value of the data point, represents the standard normalization function, represents the absolute value symbol; the path length of the data point is weighted by the product of the consistency of the data point in the isolated tree and the preference of the isolated tree, the weighted average path length of the weighted data point is processed by the isolation forest algorithm to obtain the abnormal score of the data point, and the pipeline safety warning result is obtained according to the comparison result of the abnormal score with the preset threshold.

[0010] The present invention takes into account that abnormal data with slight changes do not change significantly in the overall data. In order to accurately obtain the safety monitoring results of pipeline flow data including such data points, the present invention calculates the preference of each isolated tree and the consistency of the distribution of data points in each isolated tree with the distribution in the flow data sequence, and weights the path length by the preference and consistency of the data points, thereby increasing the sensitivity to slight abnormal data, so that the abnormal score of the data point can be accurately obtained based on this, effectively improving the accuracy of pipeline safety early warning.

[0011] According to a pipeline safety early warning method based on big data provided by the present invention, the isolation forest algorithm is used to process each data point in the flow data sequence of the pipeline, and the method also includes: deploying flow sensors at pipeline detection points to collect flow data of the pipeline; and obtaining a flow data sequence of the pipeline after preprocessing the flow data, wherein the flow data sequence includes multiple data points.

[0012] The present invention takes into account that the original data collected by the sensor is not conducive to subsequent data processing, and therefore obtains the flow data sequence through preprocessing, which can effectively improve the overall quality of the data.

[0013] According to a pipeline safety early warning method based on big data provided by the present invention, the optimization degree of the isolated tree satisfies the relationship:

[0014] ;

[0015] In the formula, represents the preference of the ith isolated tree, represents the number of data points in the ith isolated tree, represents the path length of the jth data point in the i-th isolated tree, represents the mean path length of all data points in the i-th isolated tree, represents the path length of the first data point in the i-th isolated tree, Indicates The path length of a data point in the i-th isolated tree, Represents the maximum value function.

[0016] The present invention provides an accurate calculation formula for the optimization degree of an isolated tree. The quality of the isolated tree is obtained by the standard deviation of the path length of each data point in the isolated tree. The larger the standard deviation, the higher the abnormal separation effect of the isolated tree, and the better the quality of the corresponding isolated tree. If the height of the isolated tree is high and the standard deviation of the path length is large, the optimization degree of the isolated tree is also high.

[0017] According to a pipeline safety early warning method based on big data provided by the present invention, the weighted average path length of the data points satisfies the relationship:

[0018] ;

[0019] In the formula, represents the weighted average path length of the jth data point, represents the number of isolated trees, represents the path length of the jth data point in the i-th isolated tree, represents the preference of the ith isolated tree, Represents the consistency of the j-th data point in the i-th isolated tree.

[0020] According to a pipeline safety early warning method based on big data provided by the present invention, the weighted average path length of the data points obtained after weighting by the isolation forest algorithm is used to obtain the abnormality score of the data point, including: using the weighted average path length of the data points as the average path length of the data points in the isolation forest algorithm to obtain the abnormality score of the data point.

[0021] According to a pipeline safety early warning method based on big data provided by the present invention, the pipeline safety early warning result is obtained according to the comparison result of the abnormal score and the preset threshold, including: a preset threshold; if the abnormal score of the data point is greater than the preset threshold, the early warning result of the data point is abnormal; if the abnormal score of the data point is not greater than the preset threshold, the early warning result of the data point is normal.

[0022] By presetting a reasonable threshold, the present invention can more flexibly determine the real abnormal data according to the actual situation, and effectively improve the accuracy of the obtained safety warning results.

[0023] According to a pipeline safety early warning method based on big data provided by the present invention, the early warning result of the data point is obtained as abnormal, and then it also includes: marking the data point as abnormal and issuing an abnormal prompt to the outside.

[0024] The present invention takes into account that abnormal data in pipeline flow data may pose a greater safety hazard, and therefore promptly issues an early warning after abnormal data is monitored, which can effectively reduce the possibility of accidents.

[0025] In a second aspect, the present invention provides a pipeline safety early warning system based on big data, which adopts the following technical solutions:

[0026] A pipeline safety early warning system based on big data comprises: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned pipeline safety early warning method based on big data is implemented.

[0027] By adopting the above technical solution, the above-mentioned pipeline safety early warning method based on big data is generated into a computer program and stored in a memory so as to be loaded and executed by a processor, thereby making a terminal device based on the memory and the processor for easy use.

[0028] The present invention has the following technical effects:

[0029] Based on the above technical solution, when determining the pipeline safety early warning result, the present invention processes the flow data at the pipeline through the isolation forest algorithm to realize pipeline safety monitoring; on this basis, the present invention calculates the optimization degree of each isolated tree, and the consistency of the distribution of data points in each isolated tree and the distribution in the flow data sequence, and weights the path length by the optimization degree and consistency of the data point, thereby increasing the performance of abnormal data with slight changes in the overall data, so that the abnormal score of the data point can be accurately obtained based on this, effectively improving the accuracy of pipeline safety early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0031] Figure 1 A schematic flow chart of a pipeline safety early warning method based on big data provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0034] It should be noted that during the design and operation of the pipeline gas system, various measures will be taken to ensure the smooth transmission of gas, but there will still be various unexpected failures that lead to gas leaks. There are toxic gases in the gas pipeline. Once the pipeline leaks, it may cause serious safety hazards. Therefore, it is necessary to monitor the gas data in the pipeline in real time to ensure safe operation. In order to reduce safety hazards and ensure safe operation, it is necessary to design a pipeline flow anomaly detection method so that pipeline anomalies can be captured in time, so as to deal with and reduce safety hazards.

[0035] The Isolation Forest Algorithm is an unsupervised learning algorithm that constructs an isolation tree by randomly selecting features and segmentation points to isolate abnormal data points. Even if there are irrelevant or redundant features in high-dimensional data, the Isolation Forest Algorithm can still effectively identify abnormal data points within a relatively small number of segmentation steps. Based on this, the present invention discloses a pipeline safety early warning method based on big data, which can accurately obtain pipeline safety early warning results by processing the flow data at the pipeline through the Isolation Forest Algorithm. Figure 1 As shown, Figure 1 A schematic flow chart of a pipeline safety early warning method based on big data provided by an embodiment of the present invention, the method specifically comprises the following steps S1 to S5:

[0036] S1: Process each data point in the flow data sequence of the pipeline through the isolation forest algorithm to obtain multiple isolated trees and the path length of each data point in the isolated tree.

[0037] It should be noted that in the isolation forest algorithm, abnormal data points usually have a shorter path length to the root node, so abnormal data points are more likely to be isolated. During the segmentation process, the algorithm randomly selects a value in the traffic data sequence as a segmentation point to build an isolation tree. For each isolation tree, each data point will have a path length from the root node to the leaf node.

[0038] Among them, the specific steps of processing each data point in the flow data sequence of the pipeline through the isolation forest algorithm to obtain multiple isolated trees and the path length of each data point in the isolated tree can be achieved through existing technologies, and the embodiments of the present invention are not described in detail here.

[0039] By way of example, in an embodiment of the present invention, each data point in a flow data sequence of a pipeline is processed by an isolation forest algorithm, which also includes: deploying flow sensors at pipeline detection points to collect flow data of the pipeline; and obtaining a flow data sequence of the pipeline after preprocessing the flow data, wherein the flow data sequence includes multiple data points.

[0040] Specifically, a corresponding collection frequency is set for the flow sensor, and the flow sensor collects flow data according to the collection frequency. The flow value obtained each time is used as a data point, and a flow data sequence consisting of multiple data points is obtained after preprocessing.

[0041] The acquisition frequency of the flow sensor can be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this. The preprocessing method can be data cleaning to remove invalid data, converting flow data into a format suitable for analysis, etc., which can be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0042] Each data point in the flow data sequence obtained by preprocessing corresponds to a flow value of pipeline gas, and the number of data points in each isolated tree is the same.

[0043] After obtaining multiple isolated trees in the traffic data sequence and the path length of each data point in the isolated tree based on the above steps, continue to perform the following steps.

[0044] S2: Determine the optimization degree of the isolated tree.

[0045] It should be noted that in a gas pipeline, if a gas leak occurs due to loose joints, the initial leakage may cause slight fluctuations in the flow data at the pipeline. Such fluctuations are not significant in the overall flow data. If the anomaly score of the data point is calculated directly based on the path length of each data point obtained by the isolation forest algorithm, it is impossible to distinguish between abnormal data with slight fluctuations and normal data, which ultimately leads to low accuracy of abnormal monitoring.

[0046] Based on this, the embodiment of the present invention calculates the preference of each isolated tree. The higher the preference, the higher the quality of the isolated tree. It can be given a greater weight when calculating the anomaly score, ensuring that the high-quality tree has a greater contribution to the anomaly score. The preference of the isolated tree can characterize the importance of the isolated tree in determining the anomaly score.

[0047] Specifically, in an embodiment of the present invention, when determining the preference of an isolated tree, the standard deviation of the path lengths between the data points in the isolated tree can be obtained. The larger the standard deviation, the greater the difference in the path lengths of the data points in the isolated tree, and the easier it is to separate the data points. The stronger the ability of the isolated tree to distinguish abnormal data points, the higher the corresponding preference, wherein the preference is positively correlated with the standard deviation of the path lengths of the data points in the isolated tree; then the height of the isolated tree is obtained. The higher the height of the tree, the greater the maximum value of the path lengths of the data points in the isolated tree, and the faster the isolated tree can separate abnormal data points, wherein the preference is positively correlated with the maximum value of the path lengths of the data points in the isolated tree.

[0048] For example, in the embodiment of the present invention, the preference of the isolated tree is determined, and the following relational expression may be specifically referred to:

[0049] ;

[0050] In the formula, represents the preference of the ith isolated tree, represents the number of data points in the ith isolated tree, represents the path length of the jth data point in the i-th isolated tree, represents the mean path length of all data points in the i-th isolated tree, represents the path length of the first data point in the i-th isolated tree, Indicates The path length of a data point in the i-th isolated tree, Represents the maximum value function.

[0051] In the above formula, It represents the standard deviation of the path length of the data points in the i-th isolated tree. The larger the standard deviation, the more discrete the path length of the data points is, and the more able it is to separate the abnormal data points.

[0052] It represents the maximum path length of the data points in the ith isolated tree, that is, the height of the ith isolated tree. If there is a significant difference in the path length of the data points in the ith isolated tree, the higher the height of the isolated tree, the more accurately and quickly the ith isolated tree can separate the abnormal data, the better the quality of the isolated tree, and the higher the corresponding preference.

[0053] After obtaining the optimization degree of each isolated tree based on the above steps, the following steps may be continued to further determine the consistency of each data point in the isolated tree.

[0054] S3: Obtain the historical data segments of the data points in the traffic data sequence and calculate the consistency of the data points in the isolation tree.

[0055] It should be noted that the initial stage of gas leakage may cause slight fluctuations in the flow data at the pipeline. Such fluctuations are not significant in the overall flow data and there is a possibility of misjudgment.

[0056] Based on this, in an embodiment of the present invention, in order to effectively identify such abnormal data with slight initial fluctuations, the density of data points in a local range can be analyzed to obtain the consistency of data points in the isolation tree. If the number of data points in the parent node of the node where any data point is located in the isolation tree is greater, the position of the data point in the data space is relatively isolated, and can thus be quickly isolated.

[0057] By way of example, in an embodiment of the present invention, when determining the consistency of a data point in an isolated tree, a historical data segment of the data point can be obtained in the traffic data sequence, and the absolute difference between the traffic mean of the data point and the historical data segment is used as the first synchronization indicator; the total number of data points in the parent node of the node where the data point is located in the isolated tree is obtained, and the absolute difference between the total number of data points and the first synchronization indicator is used as the second synchronization indicator to obtain the consistency of the data point in the isolated tree, and the consistency of the data point in the isolated tree is negatively correlated with the second synchronization indicator.

[0058] The length of the historical data segment can be set to 30, and the length of the historical data segment is the number of data points contained in the historical data segment; the length of the historical data segment can be set according to actual needs.

[0059] It can be understood that the data points in the historical data segment of the current data point do not include the current data point itself.

[0060] For example, in the embodiment of the present invention, the consistency of the data points in the isolation tree is determined, and the following relationship can be specifically referred to:

[0061] ;

[0062] In the formula, represents the consistency of the jth data point in the i-th isolated tree, Represents the total number of data points in the parent node of the node where the jth data point is located in the i-th isolated tree, represents the flow value of the jth data point, Indicates the number of data points in the historical data segment, Indicates the number of historical data segments The flow value of the data point, represents the standard normalization function, Represents the absolute value symbol.

[0063] In the above formula, It represents the absolute difference between the jth data point and the mean flow rate of the data points in the historical data segment, i.e. the first synchronization indicator. The larger the value, the greater the difference between the jth data point and the local difference in the historical data segment, and the greater the possibility that the data point is an abnormal data point. If the trend of the total number of data points in the parent node of the data point is consistent with the trend of the difference between its local difference (for example, the same increase and decrease), the more consistent the distribution of the data point in the current isolated tree is with the actual collection distribution, the closer the value obtained after normalization of the two is. Therefore, if That is, the smaller the value of the second synchronization index is, the higher the consistency of the j-th data point in the i-th isolated tree is.

[0064] After obtaining the consistency of each data point in each isolated tree based on the above steps, the following steps may be continued to obtain the weighted average path length of each data point.

[0065] S4: The path length of the data point is weighted by the product of the consistency of the data point in the isolated tree and the preference of the isolated tree to obtain the weighted average path length of the data point.

[0066] It should be noted that in order to accurately capture abnormal fluctuations with small changes, when determining the abnormal score of a data point, it is necessary to weight the path length of the data point according to its performance in different isolated trees. The higher the consistency between the distribution characteristics of a data point in an isolated tree with a higher degree of preference and the difference characteristics of the data point in its local historical data, the more the data point matches the distribution characteristics of the isolated tree. Based on this, the path length of each data point can be weighted to ensure that the isolated tree with a higher degree of preference and the data points with higher consistency in the isolated tree contribute more when calculating the weighted path length, thereby improving the accuracy of pipeline safety monitoring and early warning.

[0067] For example, in an embodiment of the present invention, the weighted average path length of the data points is determined, and the specific details can be referred to the following relationship:

[0068] ;

[0069] In the formula, represents the weighted average path length of the jth data point, represents the number of isolated trees, represents the path length of the jth data point in the i-th isolated tree, represents the preference of the ith isolated tree, Represents the consistency of the j-th data point in the i-th isolated tree.

[0070] In the above formula, It represents the weighting factor of the jth data point in the i-th isolated tree. The data point with higher preference has higher contribution in determining the abnormal score. In this case, if the consistency of the jth data point is also higher, it means that the distribution of the jth data point in the i-th isolated tree is more consistent with the distribution in the actual collected data, and the credibility of the data point in the isolated tree is higher.

[0071] Represents the sum of the weighted path lengths of the j-th data point in different isolated trees. It represents the sum of weighted factors of the j-th data point in all isolated trees. The sum of weighted path lengths of data points in different isolated trees is normalized by the sum of weighted factors. This can eliminate the influence of the size of isolated trees on the path length and help to uniformly process all data points in the future.

[0072] Based on the above steps, after weighting the path length of the data point using the preference of the isolation tree and the consistency of the data point in the isolation tree, the isolation forest calculation can be performed based on the weighted path length to obtain the anomaly score of the data point.

[0073] S5: The weighted average path length of the data points is processed by the isolation forest algorithm to obtain the anomaly score of the data point, and the pipeline safety warning result is obtained based on the comparison result between the anomaly score and the preset threshold.

[0074] It should be noted that the value of the weighted average path length of the data points obtained based on the above steps is the value of the average path length of the data points used in the isolation forest algorithm. By using the average path length of the data points in the isolation forest algorithm, the anomaly score of the data points can be accurately obtained.

[0075] For example, in an embodiment of the present invention, the weighted average path length of data points is processed by an isolation forest algorithm to obtain an abnormality score of the data point, including: using the weighted average path length of the data point as the average path length of the data point in the isolation forest algorithm to obtain the abnormality score of the data point.

[0076] It is understandable that the specific step of using the average path length of each data point to obtain the abnormality score of each data point in the isolation forest algorithm can be implemented by the existing technology, and the embodiment of the present invention will not be described in detail here.

[0077] After the anomaly score of each data point is obtained based on the above method, the warning result of the data point can be obtained based on the comparison result of the anomaly score of each data point with the threshold.

[0078] For example, in an embodiment of the present invention, a pipeline safety warning result is obtained according to a comparison result between an abnormality score and a preset threshold, including: a preset threshold; if the abnormality score of a data point is greater than the preset threshold, the warning result of the data point is abnormal; if the abnormality score of the data point is not greater than the preset threshold, the warning result of the data point is normal.

[0079] The threshold value may be preset as 0.8; the threshold value may be set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0080] It is understandable that if the warning result of a data point is normal, it means that the gas flow rate of the pipeline at the current data point is normal and there is no gas leakage. If the warning result of a data point is abnormal, it means that the gas flow rate of the pipeline at the collection time corresponding to the current data point is abnormal, and there may be a gas leakage. In such a case, it is necessary to issue a reminder in time so that the staff can deal with it in time.

[0081] For example, in an embodiment of the present invention, after obtaining a warning result that a data point is abnormal, the data point may be marked as abnormal and an abnormal prompt may be issued externally.

[0082] The abnormal prompt may be in the form of a sound prompt or a signal light prompt, etc., which may be specifically set according to actual needs, and the embodiment of the present invention does not impose too many restrictions on this.

[0083] In this way, abnormal situations of gas leakage can be dealt with in a timely manner, thereby effectively reducing safety hazards.

[0084] It can be seen that in an embodiment of the present invention, when determining the pipeline safety warning result, each data point in the flow data sequence of the pipeline can be processed by the isolation forest algorithm to obtain multiple isolated trees and the path length of each data point in the isolated tree; determine the preference of the isolated tree, and the preference is positively correlated with the standard deviation of the path length of each data point in the isolated tree; obtain the historical data segment of the data point in the flow data sequence, and use the absolute difference between the data point and the flow mean of the historical data segment as the first synchronization indicator; obtain the total number of data points in the parent node of the node where the data point is located in the isolated tree, and use the absolute difference between the total number of data points and the first synchronization indicator as the second synchronization indicator to obtain the consistency of the data point in the isolated tree, and the consistency is negatively correlated with the second synchronization indicator; calculate the weighted average path length of the data point; process the weighted average path length of the data point by the isolation forest algorithm to obtain the abnormal score of the data point, and obtain the pipeline safety warning result according to the comparison result of the abnormal score with the preset threshold.

[0085] In this way, the embodiment of the present invention can realize pipeline safety monitoring by processing the flow data at the pipeline through the isolation forest algorithm; on this basis, the embodiment of the present invention takes into account that the abnormal data with small changes does not change significantly in the overall data. In order to accurately obtain the safety monitoring results of the pipeline flow data including such data points, the embodiment of the present invention calculates the preference of each isolated tree and the consistency of the distribution of the data points in each isolated tree with the distribution in the flow data sequence, and weights the path length by the preference and consistency of the data points, thereby increasing the sensitivity to small abnormal data, so that the abnormal score of the data point can be accurately obtained based on this, and the accuracy of pipeline safety warning is effectively improved.

[0086] An embodiment of the present invention further discloses a pipeline safety early warning system based on big data, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a pipeline safety early warning method based on big data according to the present invention is implemented.

[0087] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0088] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM, a dynamic random access memory DRAM, a static random access memory SRAM, an enhanced dynamic random access memory EDRAM, a high bandwidth memory HBM, a hybrid memory cube HMC, etc., or any other medium that can be used to store the required information and can be accessed by an application, a module, or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0089] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0090] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A pipeline safety early warning method based on big data, characterized in that: include: Each data point in the flow data sequence of the pipeline is processed by the isolation forest algorithm to obtain multiple isolated trees and the path length of each data point in the isolated trees; Determining the optimization degree of the isolated tree, wherein the optimization degree is positively correlated with the standard deviation of the path length of each data point in the isolated tree; Obtain the historical data segment of the data point in the traffic data sequence, and calculate the consistency of the data point in the isolation tree: ; In the formula, represents the consistency of the jth data point in the i-th isolated tree, Represents the total number of data points in the parent node of the node where the jth data point is located in the i-th isolated tree, represents the flow value of the jth data point, Indicates the number of data points in the historical data segment, Indicates the historical data segment The flow value of the data point, represents the standard normalization function, Indicates the absolute value symbol; The path length of the data point is weighted by the product of the consistency of the data point in the isolated tree and the preference of the isolated tree, and the weighted average path length of the weighted data point is processed by the isolation forest algorithm to obtain the abnormal score of the data point, and the pipeline safety warning result is obtained according to the comparison result of the abnormal score with the preset threshold.

2. According to the big data-based pipeline safety early warning method of claim 1, it is characterized in that: The process of processing each data point in the flow data sequence of the pipeline by the isolation forest algorithm also includes: A flow sensor is arranged at a pipeline detection point to collect flow data of the pipeline; the flow data is preprocessed to obtain a flow data sequence of the pipeline, wherein the flow data sequence includes a plurality of data points.

3. The pipeline safety early warning method based on big data according to claim 1 is characterized in that: The optimization degree of the isolated tree satisfies the relationship: ； In the formula, represents the preference of the ith isolated tree, represents the number of data points in the ith isolated tree, represents the path length of the jth data point in the i-th isolated tree, represents the mean path length of all data points in the i-th isolated tree, represents the path length of the first data point in the i-th isolated tree, Indicates The path length of a data point in the i-th isolated tree, Represents the maximum value function.

4. The pipeline safety early warning method based on big data according to claim 1 is characterized in that: The weighted average path length of the data points satisfies the relationship: ； In the formula, represents the weighted average path length of the jth data point, represents the number of isolated trees, represents the path length of the jth data point in the i-th isolated tree, represents the preference of the ith isolated tree, Represents the consistency of the j-th data point in the i-th isolated tree.

5. The pipeline safety early warning method based on big data according to claim 1 is characterized in that: The weighted average path length of the data points obtained by weighted processing by the isolation forest algorithm to obtain the abnormality score of the data point includes: The weighted average path length of the data point is used as the average path length of the data point in the isolation forest algorithm to obtain the abnormal score of the data point.

6. The pipeline safety early warning method based on big data according to claim 1 is characterized in that: The pipeline safety warning result is obtained according to the comparison result between the abnormality score and the preset threshold, including: Preset threshold value; If the abnormal score of the data point is greater than a preset threshold, the warning result of the data point is abnormal; If the abnormal score of the data point is not greater than the preset threshold, the warning result of the data point is normal.

7. The pipeline safety early warning method based on big data according to claim 6 is characterized in that: The early warning result of obtaining the data point is abnormal, and then further comprising: After the data point is marked as abnormal, an abnormal prompt is issued externally.

8. A pipeline safety early warning system based on big data, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a pipeline safety early warning method based on big data according to any one of claims 1 to 7 is implemented.

Images