A method, device, and medium for processing abnormal data from a distributed fiber optic sensor.

Abstract

This invention relates to the field of data processing, and in particular to a method, device, and medium for processing abnormal data from a distributed fiber optic sensor. The method includes: synchronously acquiring temperature, vibration, and distance data from each monitoring point; calculating the theoretical expected temperature range based on a fiber optic sensing physical relationship model, and initially labeling monitoring points exceeding the range as candidate points for pseudo-anomalies; eliminating candidate points attributable to environmental mechanical interference through vibration-temperature coupling analysis; reconstructing the temperature values ​​of the remaining candidate points online using an adaptive residual compensation network; integrating the reconstructed temperature values ​​with normal point data to form a cleaned dataset; and based on this dataset, identifying monitoring points that conform to true anomaly characteristic patterns through spatiotemporal joint analysis and generating alarms. This invention effectively distinguishes between true and false anomalies, significantly improving the accuracy and reliability of pipeline safety monitoring.

Description

Technical Field

[0001] This invention relates to the field of data processing, and in particular to a method, device, and medium for processing abnormal data from a distributed optical fiber sensor. Background Technology

[0002] Distributed fiber optic sensing technology is playing an irreplaceable role in pipeline safety monitoring. However, in practical engineering applications, especially in long-term monitoring in complex environments, a prominent challenge arises: frequent environmental interference events caused by non-pipeline faults lead to a persistently high false alarm rate in monitoring systems.

[0003] Current mainstream anomaly detection methods, whether based on fixed threshold criteria or using machine learning models to directly classify monitoring data as anomalies, essentially treat all data points deviating from normal patterns as potential risks. These methods have inherent flaws: First, they struggle to distinguish whether the root cause of data anomalies is a genuine pipeline fault (such as leakage or excessive stress) or sensor signal distortion caused by external environmental interference such as construction machinery, traffic loads, or geological activity (which can be considered pseudo-anomalies). Second, directly inputting all anomaly data into the diagnostic model leads to the model learning a large number of irrelevant interference patterns, reducing the accuracy of identifying genuine faults and generating numerous invalid alarms, severely consuming operational resources.

[0004] While existing technologies (e.g., patent document with publication number CN119845364B) improve the completeness of anomaly detection by fusing multi-path identification results, their core remains focused on expanding the coverage and confidence of anomaly perception, without addressing the fundamental issue of distinguishing between genuine and false anomaly sources. The interpolation prediction methods they employ aim to infer the condition of unmonitored areas, rather than performing interpretable analysis and cleaning of anomaly data from known monitoring points.

[0005] Furthermore, even with more advanced signal processing or image recognition methods, existing technologies primarily focus on improving the sensitivity or efficiency of anomaly detection, without fundamentally addressing the problem of anomaly identification. For example, patent document CN114049336B discloses a method for detecting temperature anomalies in GIS casings based on infrared image processing. This method constructs a spatial temperature distribution model and a frequency distribution model, establishes dual baselines to filter interference and extract key information, and ultimately automatically marks suspected anomaly points. This approach represents the current mainstream approach: utilizing algorithms to detect anomalies more efficiently and accurately. However, it fails to answer a crucial question: is the detected temperature anomaly a sign of internal equipment failure, or a false anomaly caused by external instantaneous heat sources, solar reflection, or other environmental factors? When faced with massive, continuous linear monitoring data generated by distributed fiber optic sensing, this method not only cannot handle distance-related systematic errors but also lacks the ability to fuse and analyze multi-dimensional information such as vibration, making it unsuitable for long-distance pipeline safety monitoring scenarios with stringent requirements for false alarm rates.

[0006] Therefore, designing a data processing method that can effectively identify and eliminate false anomalies caused by environmental interference, thereby focusing on identifying true anomalies that truly characterize pipeline failures, has become a key technical bottleneck for improving the practicality of distributed fiber optic sensing systems and reducing operation and maintenance costs. Summary of the Invention

[0007] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0008] According to a first aspect of the present invention, a method for processing abnormal data from a distributed optical fiber sensor is provided, comprising the following steps:

[0009] S1, During a continuous monitoring period, synchronously acquire monitoring data from each monitoring point of the distributed optical fiber sensor. The monitoring data includes at least temperature values, vibration values, and distance values ​​between the monitoring points and the data receiving end.

[0010] S2, based on the preset optical fiber sensing physical relationship model and the distance value of each monitoring point, calculate the theoretical expected temperature range of each monitoring point in the current monitoring period; if the actual monitored temperature value of a monitoring point exceeds the theoretical expected temperature range calculated for that monitoring point, then mark the monitoring point as a pseudo-anomaly candidate point.

[0011] S3, for the pseudo-anomaly candidate point, determine whether the vibration value of the pseudo-anomaly candidate point exceeds the vibration temperature coupling reference threshold; if so, determine that the pseudo-anomaly candidate point belongs to anomalies that can be explained by environmental mechanical interference, and remove the pseudo-anomaly candidate point from the anomaly judgment of the current cycle.

[0012] S4. The remaining pseudo-anomaly candidate points after elimination are taken as candidate points to be reconstructed. The temperature value, vibration value and distance value of each candidate point to be reconstructed are input into an adaptive residual compensation network to obtain the corresponding temperature compensation value. The temperature compensation value is then used to reconstruct the temperature value of each candidate point to be reconstructed online to obtain the reconstructed temperature value.

[0013] S5. Based on the reconstructed temperature value and the original temperature values ​​of all monitoring points that were not marked as pseudo-anomaly candidates, a cleaned monitoring dataset is formed.

[0014] S6. Based on the cleaned monitoring dataset, perform spatiotemporal joint analysis on the candidate points to be reconstructed, identify monitoring points that conform to the preset real abnormal feature pattern from the candidate points to be reconstructed, and generate abnormal alarm information.

[0015] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by invoking a program or instructions stored in the memory.

[0016] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0017] The present invention has at least the following beneficial effects:

[0018] 1. This invention achieves efficient identification of true and false anomalies, fundamentally reducing the false alarm rate. Existing technologies typically treat all data deviating from the baseline as potential risks, failing to distinguish the root cause of the anomaly. This invention innovatively introduces a multi-level collaborative screening mechanism: First, through a physical relationship model of fiber optic sensing based on physical laws, a dynamic rationality judgment standard related to the transmission distance is established for each monitoring point, initially screening out suspicious points caused by systematic deviations; then, it uniquely utilizes vibration-temperature coupling analysis to determine temperature anomalies accompanied by strong vibrations as false anomalies caused by external mechanical interference and eliminates them. This two-level serial judgment mechanism of physical initial screening and coupled elimination fundamentally separates environmental interference from intrinsic faults, solving the core problem leading to false alarms.

[0019] 2. Improved quality of anomalous data, laying a reliable data foundation for accurate analysis. For remaining anomalies that cannot be explained by vibration, this invention does not simply discard them or issue an alarm, but intelligently reconstructs them through an adaptive residual compensation network. This network aims to approximate the theoretical baseline value of the physical model, learning to generate temperature compensation values, thereby reconstructing anomalous data affected by complex unmodeled factors into reconstructed values ​​that are closer to the true physical state. This step significantly improves the reliability and consistency of the dataset used for final judgment, avoiding secondary misjudgments caused by data quality issues.

[0020] 3. This invention achieves accurate identification and alarm of actual pipeline fault modes, improving the intelligence level of operation and maintenance decision-making. Based on high-quality cleaning data, this invention proposes for the first time to perform spatiotemporal joint analysis on a high-risk point set (candidate points to be reconstructed). By defining multi-dimensional feature patterns such as spatial continuity, temporal persistence, and coupling separation, it accurately captures the unique characteristics of actual faults such as minor leaks and slow overheating. This method upgrades alarms from simple threshold exceeding to complex pattern matching, enabling alarm information to not only point to the abnormal location but also indicate the nature and possible causes of the abnormality. This provides operation and maintenance personnel with direct and actionable decision support, greatly enhancing the predictability and proactivity of pipeline safety management.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating an abnormal data processing method for a distributed optical fiber sensor, as provided in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0027] This invention provides a method for processing abnormal data from a distributed optical fiber sensor, applicable to scenarios involving safety monitoring of pipelines equipped with distributed optical fiber sensors. The distributed optical fiber sensor transmits light pulses into a sensing optical fiber laid along the pipeline and receives backscattered light (such as Raman scattering, Brillouin scattering, or Rayleigh scattering) from the sensing fiber. By analyzing the characteristics of the scattered light, it obtains physical quantity information such as temperature and vibration at each monitoring point along the fiber's length.

[0028] In a typical embodiment of the present invention, the pipeline is a long-distance underground transport pipeline (such as an oil and gas pipeline), and the sensing optical fiber is usually laid in parallel along the pipeline or embedded in the pipeline structure to form a continuous distributed sensing network.

[0029] Figure 1 The diagram shows a flowchart of an abnormal data processing method for a distributed optical fiber sensor according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps: S1, synchronous data acquisition: synchronously acquire monitoring data from each monitoring point of the distributed optical fiber sensor within a continuous monitoring period.

[0030] In this embodiment of the invention, data acquisition and processing are performed in preset, continuous monitoring cycles. The duration of the monitoring cycle needs to be determined comprehensively based on the operating characteristics of the monitored fiber optic pipeline, the rate of environmental change, and the real-time requirements for anomaly response. For example, for scenarios with extremely high safety requirements, such as long-distance oil and gas pipelines, the monitoring cycle can be set to 1 minute; while for municipal integrated utility tunnels with relatively stable and slow-changing operating environments, the monitoring cycle can be set to 10 minutes or longer. Within each monitoring cycle, the distributed fiber optic sensors can be controlled by the monitoring system to complete a complete data acquisition along the pipeline.

[0031] The monitoring data includes at least the temperature value, vibration value, and fiber optic distance (or channel distance) between each monitoring point and the data receiving end (i.e., the signal demodulation and processing unit).

[0032] Temperature value: Calculated by analyzing the intensity ratio of backscattered Raman light obtained by distributed temperature sensing (DTS) technology, reflecting the absolute or relative temperature at the monitoring point.

[0033] Distance value: It is precisely calculated based on the propagation speed of the light pulse in the optical fiber and the time difference between transmission and reception, and is used to uniquely identify the physical location of the monitoring point on the optical fiber path.

[0034] Vibration value: Obtained by analyzing the phase or intensity change signal of backscattered Rayleigh light acquired through distributed acoustic sensing (DAS) or distributed vibration sensing (DVS) technologies. In a preferred embodiment of the present invention, to improve the targeting of vibration analysis and suppress noise, the vibration value is the integral value of vibration energy within a preset frequency band. Specifically, firstly, the time-domain waveform of the original vibration signal at the monitoring point is acquired, and then converted to the frequency domain using a Fast Fourier Transform (FFT); then, based on historical data analysis or prior knowledge (e.g., the main frequency range of common interferences such as external mechanical construction and vehicle traffic), one or more preset frequency bands of interest (e.g., 10Hz-100Hz) are set; finally, the integral of the power spectral density (PSD) of the vibration signal within these preset frequency bands is calculated, and this integral is used as the vibration characteristic value of the monitoring point in the monitoring period. This processing method can effectively focus on key vibration components that may cause temperature anomalies, improving the accuracy of subsequent coupling analysis.

[0035] In this invention, the meaning of synchronous data acquisition is that for the same monitoring point, its temperature value, vibration value and distance value are collected and correlated within the same time window (i.e. the same monitoring period), which ensures the consistency of different physical quantities in time and lays a reliable data foundation for subsequent multidimensional correlation analysis.

[0036] It should be understood that the above-described methods for acquiring monitoring data are merely illustrative examples. Those skilled in the art can make adaptive adjustments based on the type of sensor and technical standards used in the actual application, as long as the required temperature, vibration, and distance information can be obtained.

[0037] S2, Initial screening of pseudo-anomaly candidate points: Based on the preset optical fiber sensing physical relationship model and the distance value of each monitoring point, calculate the theoretical expected temperature range of each monitoring point in the current monitoring cycle; if the actual monitored temperature value of a monitoring point exceeds the theoretical expected temperature range calculated for that monitoring point, then the monitoring point is marked as a pseudo-anomaly candidate point.

[0038] In this invention, the preset fiber optic sensing physical relationship model is used to characterize the intrinsic relationship between the theoretical temperature value of the monitoring point and the distance from the monitoring point to the data receiving end under ideal conditions without external abnormal heat source interference. The core of the preset fiber optic sensing physical relationship model lies in the upper limit T of the expected range of the theoretical temperature. max and lower limit value T min All of them have a preset functional relationship with the distance value L, that is, T max =f max (L), T min =f min (L). Preset functional relationship f max (L) and f min (L) reflects the systematic spatial variation law of temperature readings and its reasonable fluctuation range caused by factors such as inherent transmission attenuation of optical fiber and background temperature field under the reference state without real faults.

[0039] The predetermined functional relationship can be determined in any of the following ways:

[0040] Method 1: Calibration Experiment Method

[0041] This method is implemented during the extended period when the fiber optic duct system is known to be operating normally (e.g., no leaks, no external abnormal heat sources). The specific steps are as follows:

[0042] (1) Data collection: During the extended period when the fiber optic pipeline system is known to be operating normally, historical monitoring data of all monitoring points under various environmental conditions are collected to form a modeling dataset.

[0043] Suppose the fiber optic duct system has n monitoring points, and each monitoring point i (where i ranges from 1 to n) has a fixed fiber optic distance L. i ;

[0044] At m different historical time points (or monitoring periods) j (where j ranges from 1 to m), record the temperature measurement value T at each monitoring point i. ij .

[0045] Therefore, the modeling dataset contains n×m data samples, each of which can be represented as (L i T ij ).

[0046] (2) Data analysis and function fitting:

[0047] Step 1: Aggregate data by monitoring point (distance). For each monitoring point i (i.e., each distance L) i This yields a set {T} containing m temperature values. i1 , ..., T ij , ..., Tim}

[0048] Step 2: Calculate the statistics for each distance point. Perform statistical analysis on the temperature value set for each monitoring point i, for example, calculate its sample mean μ. i and sample standard deviation σ i .

[0049] Step 3: Define the expected value range boundaries. Based on the statistical results, define the expected value range boundaries for each distance L. i Define the upper reference bound B for the range of expected theoretical temperatures. max (L) i ) and lower bound B min (L) i A typical definition is: B max (L) i )=μ i +k×σ i B min (L) i )=μ i −k×σ i Where k is a constant determined according to the required confidence level (e.g., 95%) (e.g., k=2, corresponding to a confidence interval of approximately 95%).

[0050] Step 4: Fit a continuous function. Combine the obtained n sets of data (L... i B max (L) i )) and (L i B min (L) i Using these two data points as training data, curve fitting techniques (such as linear regression, multinomial regression, spline interpolation, or exponential fitting) are used to fit two continuous functions f. max (L) and f min (L), so that they can each approximate these data points. Ultimately, these two functions f... max (L) and f min The analytical expression of (L) (or the lookup table generated therefrom) constitutes the core of the optical fiber sensing physical relationship model.

[0051] Through the above process, the model can handle any input distance value L (not necessarily exactly equal to a certain L). i Output a theoretical temperature expectation range [f] min (L), f max (L)].

[0052] Method 2: Analytical Modeling Based on Fiber Optic Transmission Theory and Thermal Environment

[0053] This approach combines the principles of fiber optic sensing with the thermodynamics of the pipeline environment for derivation:

[0054] 1. Theoretical Basis: Consider the optical power attenuation model of a distributed temperature sensing (DTS) system. The relationship between the backscattered light intensity I(L) received at a distance L and the initial light intensity I0 can be expressed as I(L) = I0·exp(-α×L), where α is the total attenuation coefficient of the optical fiber. The demodulated temperature reading T... read (L) is related to I(L) and is influenced by the ambient background temperature field T. env The influence of (L).

[0055] 2. Model Construction: Taking into account both light attenuation and thermal environment factors, the theoretical reference temperature T can be derived. base The expression for (L), for example, is of the form T. base (L) = C·exp(-β×L) + T env (L), where C and β are system-related parameters. The ambient temperature background field can be obtained through thermal calculations or simplified assumptions (such as linear distribution) based on pipeline burial depth, soil characteristics, etc.

[0056] 3. Determine the expected range: in T base Based on (L), a tolerance function ΔL is introduced to characterize system noise, measurement fluctuations, and model uncertainty. The expected range of the theoretical temperature can then be defined as: [T base (L) - △L, T base (L) + ΔL]. ΔL can be a constant or a function that varies with distance (e.g., considering that the signal-to-noise ratio increases as the distance decreases).

[0057] In practical applications, regardless of the method used, the final established model should be able to output a corresponding theoretical temperature expectation range for any given distance value. This model is calibrated and established during initialization and continuously applied in subsequent monitoring.

[0058] For any monitoring point within the current monitoring period, its distance value is read. By querying or calculating the aforementioned preset fiber optic sensing physical relationship model, its specific theoretical expected temperature range is immediately obtained and compared with the actual monitored temperature value, thus efficiently completing the initial screening of pseudo-anomaly candidate points. If the actual monitored temperature value of a monitoring point exceeds the theoretical expected temperature range calculated for that monitoring point, it indicates that the temperature reading of that monitoring point cannot be explained by the inherent systematic errors of the fiber optic sensing system and conventional environmental factors. It deviates from the reasonable data range that it should be in based on its physical location (distance). Therefore, this monitoring point needs to be marked as a pseudo-anomaly candidate point.

[0059] S3, Pseudo-anomaly elimination based on environmental coupling: For the pseudo-anomaly candidate point, determine whether the vibration value of the pseudo-anomaly candidate point exceeds the vibration temperature coupling reference threshold; if so, determine that the pseudo-anomaly candidate point belongs to anomalies that can be explained by environmental mechanical interference, and eliminate the pseudo-anomaly candidate point from the anomaly judgment of the current cycle.

[0060] In this invention, the vibration-temperature coupling reference threshold is a criterion used to determine whether a temperature anomaly is likely caused by external mechanical vibration. The core idea is that if a sufficiently strong vibration signal accompanies a temperature anomaly, the anomaly is more likely to be attributed to a momentary, external mechanical disturbance (such as construction, vehicle traffic, geological activity, etc.) rather than a persistent, real fault in the pipeline itself (such as a leak). This threshold is not a fixed value but is dynamically set based on the correlation analysis results between historical vibration data from monitoring points and temperature anomaly records.

[0061] The dynamic settings are accomplished through the following possible correlation analysis process:

[0062] (1) Historical data preparation: Collect the dataset of the monitoring point in the long-term historical monitoring, including the vibration value sequence and temperature status record at each time point (marked whether it is "abnormal", such as the historical judgment result from step S2).

[0063] (2) Correlation Modeling: Analyze the co-occurrence relationship between "strong vibration events" and "abnormal temperature events" in historical data. Specifically, a basic vibration intensity threshold V can be set. base Historical vibration values ​​exceeding V base The moments of occurrence are recorded as potential strong vibration events. Then, the proportion (or probability) P of the temperature state being recorded as "abnormal" at the same time or within a short lag time of these strong vibration events is statistically analyzed.

[0064] (3) Threshold determination: Based on P and a preset confidence level Pt, determine the current vibration temperature coupling reference threshold Vc. The logic is: find a vibration intensity value V. couple This makes the monitored vibration value V curent ≥V couple At that point, there is sufficient confidence (i.e., probability P is not less than Pt) to conclude that the current temperature anomaly is caused by vibrations of this intensity.

[0065] V couple The value of V can be obtained by fitting the relationship curve between vibration intensity and the probability of temperature anomalies, and then solving P(V) = Pt. In practical implementation, this can be simplified to: selecting the lowest vibration intensity value among historical strong vibration events whose corresponding temperature anomaly occurrence ratio exceeds Pt, as V. couple .

[0066] (4) Periodic updates: V couple It can be recalculated and updated at fixed intervals (such as monthly or quarterly) or when historical data accumulates to a certain scale to reflect changes in environmental conditions or pipeline status.

[0067] In this way, the vibration temperature coupling reference threshold can adaptively reflect the actual environmental interference level of a specific monitoring point.

[0068] When step S3 is executed, the current vibration intensity value V of the monitoring point is directly called. couple The real-time vibration values ​​of the pseudo-anomaly candidate points are compared with V. couple Comparison. If the intensity exceeds the threshold, it indicates that the vibration intensity at this monitoring point has reached a level historically highly correlated with temperature anomalies. Based on the historical correlation analysis model, there is a sufficiently high confidence level (not lower than the preset Pt) to conclude that the current temperature anomaly occurring at this monitoring point is caused by this mechanical vibration interference, rather than a genuine pipeline fault. Therefore, this point needs to be removed from the current cycle's anomaly assessment process and recorded as an environmental coupling event. This design ensures the system's efficient filtering of known interference patterns, significantly improving the accuracy of subsequent identification of genuine anomalies.

[0069] S4, Adaptive Reconstruction of Abnormal Data: The remaining pseudo-abnormal candidate points after elimination are taken as candidate points to be reconstructed. The temperature value, vibration value and distance value of each candidate point to be reconstructed are input into an adaptive residual compensation network to obtain the corresponding temperature compensation value. The temperature compensation value is then used to reconstruct the temperature value of each candidate point to be reconstructed online to obtain the reconstructed temperature value.

[0070] In this invention, the reconstruction temperature value of each candidate point to be reconstructed is equal to the original temperature value of the candidate point plus the temperature compensation value output by the adaptive residual compensation network.

[0071] In this invention, the adaptive residual compensation network is trained with the goal of minimizing the residual between the reconstructed temperature value (i.e., the original temperature value plus the temperature compensation value output by the network) and the median expected theoretical temperature calculated by the preset fiber optic sensing physical relationship model.

[0072] Furthermore, in an illustrative embodiment, the adaptive residual compensation network is a multilayer perceptron, comprising an input layer, at least one hidden layer, and an output layer. The input layer has 3 nodes, corresponding to temperature, vibration, and distance values; the output layer has 1 node, corresponding to the temperature compensation value; the number of layers and nodes in the hidden layer is set according to actual needs. Fully connected layers are used between each other, the hidden layer activation function is the ReLU function, and the output layer activation function is a linear function.

[0073] Furthermore, the training process of the adaptive residual compensation network includes:

[0074] (1) Constructing a training sample set: Select multiple sample monitoring points from historical monitoring data. Each sample monitoring point (i.e., a training sample) satisfies the following condition: its historical temperature value is within the range of the theoretical expected temperature calculated based on its distance value using the optical fiber sensing physical relationship model. This is considered a normal data point. The input features of each training sample are the temperature value, vibration value, and distance value of the sample monitoring point. The output label is the median of the theoretical expected temperature calculated based on the distance value using the preset optical fiber sensing physical relationship model. The median of the theoretical expected temperature refers to the middle value of the range of theoretical expected temperatures calculated based on the distance value of the monitoring point using the same preset optical fiber sensing physical relationship model. It is the arithmetic mean of the upper and lower limits of the corresponding theoretical expected temperature range. This median of the theoretical expected temperature serves as the theoretical temperature reference value for the optical fiber duct at this location under a non-abnormal baseline state.

[0075] (2) Training process: The training sample set is divided into a training set and a validation set according to the proportion. The network is trained using the training set, and the mean square error between the reconstructed temperature value calculated by the temperature compensation value output by the network and the label value is used as the loss function. Iterative optimization is performed using the backpropagation algorithm and the Adam optimizer. During the training process, the model performance is monitored using the validation set. Training is stopped when the validation set loss function no longer decreases within several consecutive epochs to prevent overfitting.

[0076] After training, the adaptive residual compensation network is deployed in the online system. During application, for each candidate point to be reconstructed, its temperature, vibration, and distance values ​​are input into the adaptive residual compensation network. The network then performs forward propagation to calculate and output a temperature compensation value. Finally, this temperature compensation value is added to the original temperature value of the candidate point to obtain the reconstructed temperature value.

[0077] The technical advantage of this step is that, through a well-trained adaptive residual compensation network, a reasonable temperature compensation value can be generated for candidate temperature anomalies that cannot be explained by environmental vibrations. This compensation value aims to pull anomalous temperature readings affected by unmodeled disturbances or complex factors back to a range closer to their theoretical normal values, thereby obtaining more reliable and consistent reconstructed temperature values. This reconstruction process provides a cleaned and corrected high-quality data foundation for the subsequent step S5 to identify real anomalies, effectively reducing the risk of misjudgment due to poor data quality and improving the overall method's ability to identify real pipeline faults.

[0078] S5, True Anomaly Identification: Based on the reconstructed temperature value and the original temperature values ​​of all monitoring points not marked as pseudo-anomaly candidates, a cleaned monitoring dataset is formed.

[0079] Specifically, the formation of the cleaned monitoring dataset can be achieved by maintaining a list containing all monitoring point IDs. For each monitoring point:

[0080] If its ID exists in the set of "remaining pseudo-anomaly candidate points after removal", then its corresponding reconstructed temperature value is added to the dataset.

[0081] If its ID does not exist in the above set (i.e., the point has never been marked as a pseudo-anomaly candidate point, or has been removed in step S3), then its original temperature value is added to the dataset.

[0082] Ultimately, the dataset contains a unique and calibrated or confirmed temperature value for all monitoring points in the current period, providing a complete and consistent data foundation for spatiotemporal joint analysis.

[0083] S6, alarm information generation: Based on the cleaned monitoring dataset, perform spatiotemporal joint analysis on the candidate points to be reconstructed, identify monitoring points that conform to the preset real abnormal feature pattern from the candidate points to be reconstructed, and generate abnormal alarm information.

[0084] This step, based on the cleaned monitoring dataset, performs spatiotemporal joint analysis on the candidate points to be reconstructed, identifies monitoring points that conform to a preset real anomaly characteristic pattern, and generates anomaly alarm information. In this invention, the preset real anomaly characteristic pattern refers to a pattern that satisfies at least one of the following conditions:

[0085] Spatial continuity mode: The candidate points to be reconstructed are spatially continuous or adjacent, and the reconstruction temperature values ​​exhibit a gradient change along the pipeline direction.

[0086] Analyze the spatial distribution of candidate points to be reconstructed along the fiber optic duct. If multiple candidate points are found to be spatially continuous or closely adjacent (e.g., more than three adjacent monitoring points), and their reconstructed temperature values ​​exhibit a continuous gradient change along the duct direction (e.g., temperature increase or decrease), then the spatially continuous segment is determined to satisfy this pattern. This pattern often corresponds to localized continuous heating or media leakage in the duct.

[0087] In this article, the spatially continuous segment specifically refers to a segment of the fiber optic duct consisting of multiple monitoring points that are adjacent in location and are simultaneously identified as candidate points for reconstruction.

[0088] Time-duration pattern: The same monitoring point is identified as a candidate point for reconstruction in multiple consecutive monitoring periods, and the trend of the reconstructed temperature value of the monitoring point is consistent.

[0089] Track the state of the same monitoring point over p consecutive monitoring periods (p≥3). If the monitoring point is identified as a candidate point for reconstruction over p consecutive monitoring periods, and its reconstructed temperature value sequence satisfies:

[0090] (1) Monotonicity: The sequence is strictly monotonically increasing or strictly monotonically decreasing;

[0091] (2) Gradual variability: The average rate of change of absolute temperature between adjacent periods in the sequence, V avg Less than a preset threshold (e.g., 2℃ / cycle).

[0092] Among them, V avg =[(T2-T1)+……+(T r -T r-1 )+……+(T p -T p-1 )] / p,T r To represent the reconstructed temperature value of the monitoring point in the r-th monitoring cycle, r takes values ​​from 1 to p.

[0093] If the above conditions are met, the monitoring point is determined to have experienced a sustained, slowly changing anomaly. This model helps to detect latent faults that develop slowly but continue to worsen.

[0094] Coupling-separation mode: The vibration characteristic value of the candidate point to be reconstructed is lower than the vibration anomaly threshold, while the reconstruction temperature value of the candidate point to be reconstructed exceeds the temperature anomaly threshold.

[0095] For candidate reconstruction points, both their vibration characteristic value and reconstruction temperature value are checked. If the vibration characteristic value of a point is lower than a preset vibration anomaly threshold, but its reconstruction temperature value exceeds a preset temperature anomaly threshold T... th A certain margin (e.g., exceeding T) th If the temperature reaches 10% or 5℃, this mode is considered to be met. This mode eliminates vibration interference and highlights pure thermal anomalies in a silent state, which is key to identifying internal faults.

[0096] The vibration anomaly threshold and the temperature anomaly threshold are determined based on statistical analysis of historical environmental disturbance event samples and historical actual failure event samples, respectively. Specifically, the vibration anomaly threshold is usually set according to the statistical upper limit of vibration intensity in historical environmental disturbance events (such as the 95th percentile); the temperature anomaly threshold is determined based on the statistical lower limit of temperature anomalies in historical actual failure event samples, combined with the upper limit of pipeline safe operation.

[0097] When a monitoring point is identified that meets any of the above-mentioned true anomaly characteristic patterns, a structured anomaly alarm message is automatically generated. This message includes at least: the unique identifier (ID) and geographical location of the anomaly monitoring point, the pipeline segment to which it belongs, the type of anomaly mode triggered, the current reconstructed temperature value, the degree to which the threshold is exceeded, the anomaly start time, and the duration. The alarm message is pushed to maintenance personnel in real time through multiple channels such as the monitoring system interface, SMS, or email, providing accurate decision support for quickly locating and handling real pipeline risks.

[0098] It should be understood that the abnormal data processing method provided by this invention is universal. Its core technology lies in processing and logically judging the input temperature value sequence, vibration characteristic value sequence, and their corresponding distance information generated by distributed fiber optic sensors. When applied to different specific monitoring scenarios, those skilled in the art only need to perform targeted calibration or training of the key parameters and criteria involved in the method based on the physical characteristics and historical data of that scenario. For example:

[0099] When applied to long-distance oil and gas pipeline leakage monitoring, the temperature gradient change along the pipeline in the spatial continuity mode can be attributed to the change in local temperature field caused by the leakage of the medium.

[0100] When applied to early warning of external force damage to communication optical cables, the same pattern can be interpreted as an abnormal thermal-mechanical coupling caused by events such as construction and excavation.

[0101] Through this scenario-based adaptation of parameters, the method framework constructed by this invention can flexibly and effectively serve various distributed fiber optic sensing applications, ensuring the consistency of core innovative logic while taking into account detection accuracy and reliability in specific scenarios. Furthermore, the method provided in this embodiment of the invention also includes the following steps:

[0102] S7, the abnormal alarm information, along with the corresponding monitoring point location, reconstructed temperature value, and vibration value, are simultaneously sent to the pipeline maintenance decision system; based on the received information, the pipeline maintenance decision system automatically generates or recommends corresponding inspection tasks or maintenance operation instructions.

[0103] In an Industrial Internet of Things (IIoT) architecture, this method serves as the core of the intelligent sensing and analysis layer, integrating with the upper-layer pipeline maintenance decision-making system (which can be a standalone software platform or functional module) through a standard data interface (such as an API). When this method generates an anomaly alarm, it not only pushes an alarm notification but also sends a complete multi-dimensional anomaly event package to the decision-making system. This event package includes at least:

[0104] Event identifier: unique event ID, trigger time, and exception mode type (spatial continuum / temporal continuum / coupling separation).

[0105] Anomaly details: Monitoring point ID, geographical location (such as GPS coordinates, pipeline chain number), pipe section, current reconstructed temperature value, historical temperature curve, and current vibration characteristic value.

[0106] Decision support data: specific values ​​exceeding the threshold, duration of the anomaly, status of adjacent points, and environmental parameters (optional).

[0107] Upon receiving an event packet, the pipeline maintenance decision-making system automatically performs at least one of the following operations based on its built-in expert knowledge base, historical maintenance records, and preset decision rule tree:

[0108] Generate inspection tasks: Based on the anomaly pattern and location, automatically generate and dispatch targeted on-site inspection work orders, specifying the inspection route, key points of inspection (such as leak detection and temperature detection), and recommended equipment to be carried.

[0109] Recommended maintenance instructions: Based on the severity and pattern of the anomaly, recommend preliminary emergency response or maintenance operations (such as "It is recommended to close the upstream valve and arrange for leak detection" or "It is recommended to conduct a key insulation layer inspection on the XX pipe section").

[0110] Risk assessment and early warning escalation: Based on the pipeline's transport medium, pressure, and other operating parameters, assess the potential risk level of the abnormal event and decide whether to escalate the alarm or activate the emergency plan.

[0111] This step achieves a complete closed loop from data perception to intelligent analysis to decision support, efficiently transforming accurate anomaly detection results into executable operation and maintenance actions, greatly improving the intelligence level and response efficiency of pipeline safety management.

[0112] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0113] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0114] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0115] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for processing abnormal data from a distributed optical fiber sensor, characterized in that, Includes the following steps: S1, During a continuous monitoring cycle, synchronously acquire monitoring data from each monitoring point of the distributed optical fiber sensor. The monitoring data includes at least temperature value, vibration value, and distance value between the monitoring point and the data receiving end. S2, based on the preset optical fiber sensing physical relationship model and the distance value of each monitoring point, calculate the theoretical expected temperature range of each monitoring point in the current monitoring period; if the actual monitored temperature value of a monitoring point exceeds the theoretical expected temperature range calculated for that monitoring point, then mark the monitoring point as a pseudo-anomaly candidate point. S3, for the pseudo-anomaly candidate point, determine whether the vibration value of the pseudo-anomaly candidate point exceeds the vibration temperature coupling reference threshold; if so, determine that the pseudo-anomaly candidate point belongs to anomalies that can be explained by environmental mechanical interference, and remove the pseudo-anomaly candidate point from the anomaly judgment of the current cycle. S4. The remaining pseudo-anomaly candidate points after elimination are taken as candidate points to be reconstructed. The temperature value, vibration value and distance value of each candidate point to be reconstructed are input into an adaptive residual compensation network to obtain the corresponding temperature compensation value. The temperature compensation value is then used to reconstruct the temperature value of each candidate point to be reconstructed online to obtain the reconstructed temperature value. S5. Based on the reconstructed temperature value and the original temperature values ​​of all monitoring points that were not marked as pseudo-anomaly candidates, a cleaned monitoring dataset is formed. S6. Based on the cleaned monitoring dataset, perform spatiotemporal joint analysis on the candidate points to be reconstructed, identify monitoring points that conform to the preset real abnormal feature pattern from the candidate points to be reconstructed, and generate abnormal alarm information.

2. The method according to claim 1, characterized in that, In the preset optical fiber sensing physical relationship model, the upper and lower limits of the theoretical temperature expectation range are both related to the distance value by a preset function. The preset function relationship is determined by calibration experiments or optical fiber transmission attenuation theoretical models.

3. The method according to claim 1 or 2, characterized in that, The vibration-temperature coupling reference threshold is dynamically set based on the correlation analysis results of historical vibration data and temperature anomaly records at the monitoring points.

4. The method according to claim 1 or 2, characterized in that, The adaptive residual compensation network is trained with the goal of minimizing the residual between the reconstructed temperature value and the median expected theoretical temperature calculated by the preset fiber optic sensing physical relationship model.

5. The method according to claim 4, characterized in that, The training process of the adaptive residual compensation network includes: Construct a training sample set, where the input features of each training sample are the temperature value, vibration value, and distance value of a sample monitoring point, and the output label is the expected median theoretical temperature calculated based on the distance value through a preset fiber optic sensing physical relationship model. The adaptive residual compensation network is iteratively trained by using the mean square error between the temperature compensation value output by the adaptive residual compensation network and the corresponding label value as the loss function.

6. The method according to claim 1 or 2, characterized in that, The preset real anomaly feature pattern refers to a pattern that satisfies at least one of the following conditions: Spatial continuity mode: The candidate points to be reconstructed are spatially continuous or adjacent, and the reconstruction temperature values ​​exhibit a gradient change along the pipeline direction; Time-duration pattern: The same monitoring point is identified as a candidate point for reconstruction in multiple consecutive monitoring periods, and the trend of the reconstructed temperature value of the monitoring point is consistent. Coupling-separation mode: The vibration characteristic value of the candidate point to be reconstructed is lower than the vibration anomaly threshold, while the reconstruction temperature value of the candidate point to be reconstructed exceeds the temperature anomaly threshold.

7. The method according to claim 6, characterized in that, The vibration anomaly threshold and the temperature anomaly threshold are determined based on statistical analysis of historical environmental disturbance event samples and historical real fault event samples, respectively.

8. The method according to claim 1, characterized in that, It also includes the following steps: S7, the abnormal alarm information, along with the corresponding monitoring point location, reconstructed temperature value, and vibration value, are simultaneously sent to the pipeline maintenance decision system; based on the received information, the pipeline maintenance decision system automatically generates or recommends corresponding inspection tasks or maintenance operation instructions.

9. An electronic device, characterized in that, Including processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 8 by invoking programs or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 8.

Images