Oil pipeline quality monitoring and evaluating method and system based on data processing

Abstract

The invention discloses an oil pipeline quality monitoring and evaluating method and system based on data processing, and belongs to the technical field of data processing.The method comprises the steps that sensor information is obtained, and the sensor information comprises vision, sound, temperature and vibration data; acquiring application scene information; selecting a fusion mode in combination with application scene information through early fusion, middle fusion and late fusion strategies; computing and storage resources are deployed at edge nodes close to a data source by using an edge computing method, and the edge nodes are responsible for preliminary data processing and analysis work to reduce data transmission delay; analyzing sensor information by using deep learning and machine learning algorithms, and identifying an abnormal mode and a potential fault; edge computing and cloud computing are combined to form a cloud-edge collaborative architecture, and the cloud-edge collaborative architecture is used for carrying out complex data processing and model training work. The method has the effects of reducing data transmission delay and improving the real-time performance of oil pipeline quality monitoring.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method and system for monitoring and evaluating oil pipeline quality based on data processing. Background Art

[0002] The oil pipeline system, that is, the pipeline system used to transport oil and petroleum products, is mainly composed of oil pipelines, oil transfer stations and other auxiliary related equipment. It is one of the main equipment in the oil storage and transportation industry, and is also the most important transportation equipment for crude oil and petroleum products. Compared with railway and road oil transportation, which are also land transportation methods, pipeline oil transportation has the characteristics of large transportation volume, good sealing, low cost and high safety factor.

[0003] Traditional oil pipeline quality monitoring systems typically utilize a centralized cloud computing architecture, requiring all sensor data to be transmitted to the cloud for processing and analysis. Pipeline quality monitoring requires high real-time performance and rapid response to abnormalities (such as leaks, corrosion, and vibration). Due to data transmission and processing delays, this traditional architecture makes it difficult to detect and respond to issues promptly, potentially leading to safety hazards or financial losses. Summary of the Invention

[0004] In order to reduce data transmission delay and improve the real-time performance of oil pipeline quality monitoring, the present application provides an oil pipeline quality monitoring and evaluation method and system based on data processing.

[0005] This application provides a data processing-based oil pipeline quality monitoring and assessment method and system that adopts the following technical solutions:

[0006] In a first aspect, the present application provides a method for monitoring and evaluating oil pipeline quality based on data processing, comprising the following steps:

[0007] Acquiring sensor information, including visual, sound, temperature, and vibration data;

[0008] Obtain application scenario information;

[0009] Through early fusion, mid-term fusion and late fusion strategies, the fusion method is selected in combination with application scenario information;

[0010] Using edge computing methods, computing and storage resources are deployed on edge nodes close to data sources. These edge nodes are responsible for preliminary data processing and analysis, reducing data transmission delays.

[0011] Analyzing the sensor information using deep learning and machine learning algorithms to identify abnormal patterns and potential faults;

[0012] The edge computing is combined with cloud computing to form a cloud-edge collaborative architecture, which is used to perform complex data processing and model training.

[0013] Furthermore, the edge computing is combined with digital twins, specifically including:

[0014] When edge computing resources are limited, lightweight modeling technology is used to build a simplified twin model that adapts to the edge computing environment. This edge computing enables the simplified twin model to reflect the status changes of each component of the oil pipeline system in real time, achieving accurate monitoring and rapid response to complex systems.

[0015] Adopting a physical model based on discrete element model and a data-driven model based on deep Boltzmann machine, a MIF model is constructed through a multi-layer feedforward neural network to achieve adaptive update of oil pipeline system performance degradation;

[0016] Real-scene 3D visualization is performed based on the temporal GIS system, and integrated detection is performed in combination with modern remote sensing technology.

[0017] Furthermore, after the step of obtaining sensor information, the method further includes:

[0018] Clean and preprocess the data;

[0019] Performing HIS transformation and wavelet decomposition on the vibration data and temperature data to extract joint features;

[0020] Query historical data and key tasks from the preset database;

[0021] Design priority queues to ensure that critical tasks occupy more than 80% of computing resources;

[0022] Building a digital twin model based on the historical data and machine learning algorithms, suitable for complex and difficult-to-model oil pipeline systems;

[0023] Verify the validity of the unit-level model of the digital twin model and ensure the high fidelity of the basic unit-level model;

[0024] On the basis of ensuring the validity of the unit-level model of the digital twin model, further verify the assembled or fused digital twin model;

[0025] Through simulation experiments, the performance of the digital twin model in different scenarios is tested and optimized based on the simulation results;

[0026] The effectiveness of the digital twin model is regularly monitored using a K-nearest neighbor classifier and control chart methods.

[0027] Furthermore, the step of using edge computing methods to deploy computing and storage resources on edge nodes close to data sources specifically includes:

[0028] Build a complete edge computing environment, including hardware devices, applications, operating systems, and software stacks, to achieve resource allocation and task scheduling;

[0029] Using container technology to package the application and its dependencies into a container image;

[0030] Deploy edge computing services on cloud platforms and implement dynamic resource allocation and scheduling through virtualization technology to meet large-scale and complex data processing needs;

[0031] Deploy edge computing services at the edge of the network to utilize network resources, expand coverage, and increase data transmission rates;

[0032] Deploy micro data centers at the edge of the network to disperse data processing and storage to various nodes, enabling distributed data processing.

[0033] Combining cloud and edge resources to enable data processing and analysis through a cloud-edge collaborative architecture, with the cloud collecting and preprocessing requests and the edge performing real-time computation and response, optimizing the quality and personalization of AI-generated content.

[0034] Optimize the model through pruning, quantization, and distillation techniques to adapt to the computing power resources of edge devices;

[0035] Scheduling and allocation are performed based on the complexity, priority, and resource requirements of the task, and the task is assigned to the nearest edge node for processing to ensure that the resources of the edge node are fully utilized.

[0036] Furthermore, the step of analyzing the sensor information using deep learning and machine learning algorithms to identify abnormal patterns and potential faults further includes:

[0037] extracting key features based on the sensor information;

[0038] Modeling the key features using a machine learning algorithm to predict the remaining useful life of the oil pipeline system;

[0039] Identify potential failures in the oil pipeline system by analyzing the output of the digital twin model;

[0040] Combined with the output of the digital twin model, the remaining service life of the equipment is predicted.

[0041] Furthermore, a mathematical model is constructed based on the characteristics of pipeline failures to predict the losses and impacts that may be caused by pipeline failures, including:

[0042] Establish a distribution function for the probability of pipeline failure, combine the pipeline failure intensity and loss function, and calculate the risk value;

[0043] Fuzzy set theory is used to describe the uncertainty of pipeline failures and fuzzy membership functions are constructed for risk assessment.

[0044] Using neural networks to handle nonlinear relationships in pipeline fault assessment;

[0045] Query historical pipeline failure data from the preset database;

[0046] Based on historical pipeline failure data, the frequency analysis method is used to estimate the probability of pipeline failure within a specific time period, and then a pipeline failure probability distribution model is established;

[0047] A multiple linear regression model is used to analyze the relationship between the sensor information and the probability of pipeline failure.

[0048] Furthermore, the parameters of the mathematical model are set and verified in pipeline failure analysis, including:

[0049] Determine the parameters that need to be set based on the specific needs of pipeline fault analysis;

[0050] The Bayesian model selection method is used to determine the relationship between pipeline faults and different mathematical model parameters, and to define the mathematical model space dimension and initial probability distribution;

[0051] Use Gibbs sampling to perform posterior estimation of parameters, and reduce the autocorrelation of the sampling sequence by multiple iterations and discarding some samples;

[0052] Calculate the mean and standard deviation of the posterior distribution to determine the acceptability of the parameters;

[0053] Use historical pipeline failure data to validate the mathematical model to ensure that the model's predictions are consistent with actual data;

[0054] Compare the simulation results with the actual accident records and conduct statistical analysis to ensure that the output results of the mathematical model are consistent with the actual data.

[0055] In a second aspect, the present application provides an oil pipeline quality monitoring and evaluation system based on data processing, comprising:

[0056] The perception layer is used to perceive physical entities and their operating environment in real time, and collect status data of physical entities through sensors and IoT devices;

[0057] Data layer, used for data collection, storage, processing and transmission;

[0058] The modeling layer is used to transform the data of physical entities into virtual models through mechanism modeling and data-driven modeling;

[0059] The interactive layer is used to provide user interface, visualization, and report output functions;

[0060] The decision support layer is used to realize automated decision support of the digital twin system through intelligent optimization algorithms and data mining technology. It uses historical data and real-time data to analyze, predict future trends, and provide support for decision-making;

[0061] Security assurance layer, including data encryption, access control, security auditing, and disaster recovery measures, to ensure system security and stability;

[0062] Platform software and mechanism analysis layer, including cloud computing platform, edge computing, stream computing, and in-memory computing technologies, to improve data processing efficiency and real-time performance;

[0063] The multi-source data fusion layer involves the integration of multiple types of data, including environmental data, maintenance data, and operation data, to ensure the comprehensiveness and accuracy of the data.

[0064] In a third aspect, the present application provides an intelligent terminal comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned oil pipeline quality monitoring and evaluation method based on data processing.

[0065] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned oil pipeline quality monitoring and evaluation method based on data processing.

[0066] In summary, compared with the prior art, the above technical solution has the following beneficial effects:

[0067] The data processing-based oil pipeline quality monitoring and assessment method described in this application avoids the time consumption of transmitting all sensor data (such as visual, sound, temperature, vibration, etc.) to the cloud by performing preliminary data processing and analysis at edge nodes close to the data source. The edge nodes can process sensor data in real time, quickly identify abnormal patterns (such as leaks, corrosion, vibration anomalies, etc.), and promptly issue alarms or take control measures. The edge nodes can complete data processing and anomaly detection within milliseconds to seconds, meeting the high real-time requirements of oil pipeline quality monitoring. Since data does not need to be transmitted to the cloud over long distances, edge computing significantly reduces data transmission latency, making it particularly suitable for oil pipeline monitoring scenarios with poor network conditions or in remote areas. By analyzing sensor data in real time, the system can quickly issue warnings in the early stages of a fault, avoiding safety hazards and economic losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flow chart of a method for monitoring and evaluating oil pipeline quality based on data processing according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] The present application is further described in detail below in conjunction with all the accompanying drawings.

[0070] The present invention discloses a method and system for monitoring and evaluating the quality of oil pipelines based on data processing. Figure 1 , a method for monitoring and evaluating oil pipeline quality based on data processing includes:

[0071] S101: Acquire sensor information.

[0072] Specifically, the monitoring system acquires sensor information, including visual, acoustic, temperature, and vibration data. Various sensors are deployed along the oil pipeline. Visual sensors (such as cameras) are used to monitor visual anomalies such as corrosion and cracks on the pipeline surface. Acoustic sensors (such as microphones or acoustic emission sensors) are used to capture abnormal sounds inside the pipeline (such as leaks and vibrations). Temperature sensors are used to monitor temperature changes on the pipeline surface or within the fluid. Vibration sensors are used to detect mechanical vibrations or abnormal vibrations in the pipeline. Sensors are evenly distributed at key locations along the pipeline (such as elbows, joints, and valves). By acquiring multi-source heterogeneous data (visual, acoustic, temperature, and vibration data), comprehensive data support is provided for subsequent analysis and fault identification. Real-time data collection ensures the timeliness and continuity of the monitoring system.

[0073] S102: Obtain application scenario information.

[0074] Specifically, the monitoring system defines different application scenarios based on the pipeline's operating environment (e.g., land, seabed, desert, etc.) and operating status (e.g., normal operation, shutdown, maintenance, etc.). This information is obtained through manual input or automated systems (e.g., geographic information systems, operational log systems). This information then informs the selection of subsequent data fusion strategies, ensuring that data processing methods are adapted to the needs of different application scenarios.

[0075] S103: Select a fusion method based on application scenario information.

[0076] Specifically, the monitoring system selects a fusion method based on application scenario information through early fusion, mid-term fusion, and late fusion strategies. Early fusion directly fuses multi-source data during the data acquisition phase (such as combining visual and temperature data), which is suitable for scenarios requiring high real-time performance (such as leak detection). Mid-term fusion fuses data during the feature extraction phase (such as combining the features of sound and vibration data), which is suitable for scenarios requiring a balance between real-time performance and accuracy (such as vibration anomaly detection). Late fusion fuses data during the decision-making phase (such as the analysis results of comprehensive visual, sound, temperature, and vibration data), which is suitable for scenarios requiring high precision (such as pipeline health assessment). The monitoring system dynamically selects a fusion method based on application scenario information. For example, early fusion is selected in scenarios with high real-time requirements, and late fusion is selected in scenarios with high accuracy requirements. This improves the efficiency and accuracy of data processing and ensures that the fusion strategy can adapt to the needs of different scenarios.

[0077] S104: Use edge computing methods to deploy computing and storage resources at edge nodes close to data sources.

[0078] Specifically, the monitoring system uses edge computing methods to deploy computing and storage resources at edge nodes close to the data source. The edge nodes are responsible for preliminary data processing and analysis, reducing data transmission delays. The monitoring system deploys edge computing devices (such as industrial computers and embedded devices) along the oil pipeline. The edge devices should have certain computing and storage capabilities and be able to run data processing and analysis algorithms. Edge nodes are responsible for preliminary data processing and analysis, including data preprocessing and real-time analysis (such as anomaly detection and fault warning). Only the processed results or aggregated data are transmitted to the cloud, reducing the amount of data transmitted. This in turn reduces data transmission delays, improves the real-time performance of the system, reduces the cloud computing load, and optimizes resource utilization.

[0079] S105: Analyze the sensor information using deep learning and machine learning algorithms.

[0080] Specifically, the monitoring system uses deep learning algorithms (such as convolutional neural networks and recurrent neural networks) to process visual and sound data, and machine learning algorithms (such as support vector machines and random forests) to process temperature and vibration data. Models are trained in the cloud using historical data and deployed to edge nodes. The edge nodes use the deployed models to analyze sensor data in real time, identifying abnormal patterns and potential faults. This improves the accuracy and reliability of fault identification, enables intelligent monitoring, and reduces manual intervention.

[0081] S106. Combine the edge computing with cloud computing to form a cloud-edge collaborative architecture.

[0082] Specifically, the monitoring system combines edge computing with cloud computing to form a cloud-edge collaborative architecture, which is used for complex data processing and model training. Edge nodes are responsible for tasks with high real-time requirements (such as data preprocessing and simple analysis), while the cloud is responsible for complex tasks (such as large-scale data processing and model training). Edge nodes regularly transmit processed data to the cloud for model optimization and long-term trend analysis. The edge nodes and the cloud work together to achieve global monitoring and optimization.

[0083] Furthermore, after S101, as another implementation manner, the embodiment of the present application may further include the following steps:

[0084] S201: Clean and preprocess the data.

[0085] Specifically, the monitoring system performs data cleaning operations to remove noise, outliers, and missing values from sensor data. It then smoothes the data using filtering algorithms (such as low-pass filtering and median filtering). Data preprocessing then normalizes or standardizes the data to meet the input requirements of subsequent algorithms. Time series data is aligned and interpolated to ensure temporal consistency. This improves data quality, reduces noise interference with subsequent analysis, and ensures a uniform data format for feature extraction and model training.

[0086] S202 : Perform HIS transformation and wavelet decomposition on the vibration data and temperature data to extract joint features.

[0087] Specifically, the monitoring system performs a Hue-Intensity-Saturation (HIS) transform on vibration and temperature data, converting the data from the time domain to the frequency domain and extracting frequency domain features (such as spectral energy and dominant frequency components). Wavelet transforms are used to perform multi-scale decomposition of the vibration and temperature data, extracting features at different scales. Appropriate wavelet basis functions and decomposition levels are selected, and features extracted from the HIS transform and wavelet decomposition are combined to form a joint feature vector. By extracting multi-dimensional features from vibration and temperature data, the data's expressiveness is enhanced, providing high-quality feature input for subsequent fault identification and model training.

[0088] Aiming at the multimodal characteristics of vibration and temperature data, a HIS joint feature space is designed: H (Hilbert marginal spectrum feature) extracts the instantaneous frequency energy distribution of the vibration signal through Hilbert transform; I (Intensity, intensity feature) calculates the time domain statistical intensity of the temperature signal (such as root mean square RMS); S (Synergy, collaborative feature): quantifies the dynamic correlation between vibration and temperature data (such as sliding window mutual information).

[0089] First, perform Hilbert transform on the vibration signal x(t) to obtain the analytical signal z(t) = x(t) + j·H(x(t)), calculate the instantaneous amplitude A(t) = |z(t)| and the instantaneous frequency Among them, arg(z(t)) is the argument (phase) of the complex number z(t), generating the Hilbert marginal spectrum, and converting the time-frequency energy A 2 (t) is integrated according to the frequency interval (such as 0-500Hz) to form the frequency domain energy distribution feature vector H∈Rn. Then, the statistics of the temperature signal T(t) within the sliding window (such as a 10-second window) are calculated: the root mean square Among them, N is the total number of samples, Ti is the instantaneous value of the i-th sample. Finally, in the synchronization time window, the mutual information of the vibration signal and the temperature signal is calculated, and the dynamic time warping (DTW) distance S is added. DTW , measure the temporal morphological correlation between the two, and form a collaborative feature vector S∈R 2 .

[0090] The monitoring system selects a wavelet basis based on signal characteristics. In this example, Daubechies 4 (db4) is used for vibration data to match impact characteristics, and Symlet 3 (sym3) is used for temperature data to accommodate smooth variations. The monitoring system determines the number of decomposition layers based on the Nyquist theorem. For example, if the vibration signal has a maximum frequency of 500 Hz and a sampling rate of 1 kHz, the maximum number of layers is J = log2(1000 / 500) = 1, resulting in a decomposition of five layers to cover multi-scale features.

[0091] Among them, the multi-scale decomposition and coefficient extraction operations specifically include: vibration signal decomposition [cA J , cD J , cD J-1 , ..., cD1] = DWT(x(t)), where cA J is the J-th layer approximation coefficient, cD j is the detail coefficient of the jth layer. The temperature signal is decomposed into approximate coefficients and detail coefficients in the same way.

[0092] Wavelet feature extraction requires calculating the energy E of each layer coefficient j =∑|cD j | 2 , forming an energy distribution vector; calculating the Shannon entropy of the wavelet coefficients of each layer, extracting the mean, variance, and kurtosis of the coefficients of each layer to form a statistical feature vector.

[0093] Then, the eigenvector of HIS transformation is concatenated with the vibration and temperature characteristics of wavelet decomposition (such as energy and entropy of each layer) by dimension to form a high-dimensional joint eigenvector F∈Rd . Principal component analysis (PCA) or linear discriminant analysis (LDA) is used to reduce the dimensionality of high-dimensional features, retaining the principal components with a 95% variance contribution rate. Recursive feature elimination (RFE) is used to select the top-k features that are sensitive to fault classification and reduce redundant information. In this embodiment, Python's PyWavelets library is used for wavelet decomposition, scipy.signal implements the Hilbert transform, and sklearn handles feature dimensionality reduction.

[0094] S203: Query historical data and key tasks from a preset database.

[0095] Specifically, the database is pre-built by the user to store sensor data, fault records, operation logs, and other information. Based on the current application scenario and task requirements, the monitoring system queries the database for relevant historical data and extracts the data required for key tasks (such as leak detection and corrosion assessment).

[0096] S204. Design a priority queue.

[0097] Specifically, the monitoring system categorizes tasks into critical tasks (such as real-time fault detection) and non-critical tasks (such as long-term trend analysis). It allocates computing resources using priority scheduling algorithms (e.g., shortest job first, highest response ratio first). This ensures that critical tasks receive priority computing resources, occupying more than 80% of the total resources. This improves the real-time performance and response speed of critical tasks.

[0098] S205. Establish a digital twin model based on the historical data and the machine learning algorithm.

[0099] Specifically, the monitoring system uses LSTM or Transformer encoders to capture temporal dependencies such as vibration and temperature; uses a fully connected network (DNN) to process static features (such as pipe material and wall thickness) to realize non-temporal data processing; and uses mid-term fusion (Mid-Fusion) to splice temporal and non-temporal features in the intermediate layer and then input them into the downstream task network.

[0100] Training samples are generated using a sliding window (e.g., a 24-hour window with a step size of 1 hour) and Gaussian noise is added to simulate sensor errors. A base model is trained on a common industrial equipment dataset (e.g., the NASA turbine dataset) to learn common failure modes. The underlying encoder is then frozen, and only the top-level regression and classification heads are trained to adapt to the characteristics of oil pipelines. Bayesian optimization is used to search for the optimal parameter combination: LSTM hidden layer dimension (32-512); learning rate (1e-5 to 1e-3); and dropout rate (0.1-0.5).

[0101] By deploying a lightweight inference model (TensorRT-optimized TensorFlow Lite model) at the edge, health status is calculated in real time. The complete model runs in the cloud, performing high-precision simulations (such as ANSYS-coupled physical model corrections). New data is cached at the edge and regularly uploaded to the cloud, where the Elastic Weight Consolidation (EWC) algorithm is used to update the model and avoid catastrophic forgetting.

[0102] Using historical data (such as sensor data and fault records) as a training set, appropriate machine learning algorithms (such as deep learning and support vector machines) are selected to build a digital twin model. The model is trained and its parameters optimized on the cloud or a high-performance computing platform. This creates a highly accurate digital twin model that simulates the operational status of the oil pipeline and provides digital support for the monitoring and optimization of complex systems.

[0103] S206. Verify the validity of the unit-level model of the digital twin model.

[0104] Specifically, the monitoring system verifies the basic units of the digital twin model (such as pipeline segments and sensor nodes), uses test data to evaluate the accuracy and robustness of the model, and ensures the high fidelity of the unit-level model by comparing actual data and model output.

[0105] S207. Further verify the assembled or fused digital twin model.

[0106] Specifically, the monitoring system further verifies the assembled or fused digital twin model on the basis of ensuring the validity of the unit-level model of the digital twin model; the monitoring system assembles or fuses the unit-level model into a complete digital twin model, uses actual data to verify the performance of the assembled model, adjusts the model parameters, and optimizes the overall performance, thereby ensuring that the assembled digital twin model can accurately simulate the entire oil pipeline system.

[0107] S208. Test the performance of the digital twin model in different scenarios through simulation experiments, and optimize it based on the simulation results.

[0108] Specifically, the monitoring system builds a basic scenario library: normal transportation scenario, for example, rated flow (such as 2000m 3 / h), standard pressure (6MPa), ambient temperature (-20℃~40℃); Fault scenarios: leakage scenario (aperture 1-50mm, random location distribution); corrosion scenario (pitting depth 1-10mm, area 0.1-5m 2 ); third-party damage (mechanical vibration impact, soil settlement).

[0109] Then, a parameterized script (Python / ANSYS APDL) was used to generate scenario parameter combinations, such as Latin Hypercube Sampling (LHS) to generate 1,000 combinations of corrosion depth, location, and area. ANSYS Fluent was used to simulate the flow field within the pipeline (using the k-epsilon turbulence model) to calculate the pressure gradient changes during leakage. ABAQUS was used to simulate the stress concentration effects in the corrosion area, outputting vibration mode and deformation data. COMSOL Multiphysics was used to calculate the interaction between the temperature field and fluid flow.

[0110] A co-simulation framework was built based on the FMI (Functional Mock-up Interface) standard, encapsulating the digital twin model as an FMU (Functional Mock-up Unit). This enabled real-time data exchange with ANSYS / COMSOL via TCP / IP (e.g., synchronizing pressure and vibration data once per second). Simulation output data was converted to the Apache Parquet format, containing timestamps, spatial coordinates (x, y, z), and physical quantities (pressure, temperature, and flow rate). Metadata also included the scene type (e.g., scene_type = leakage_20mm).

[0111] Connect a real-world PLC controller to a simulation system (such as dSPACE SCALEXIO) to receive virtual sensor signals from the twin model in real time. Compare the deviation between virtual control commands and actual device responses to calibrate simulation accuracy. Use an Industrial IoT platform (such as PTC ThingWorx) to establish a mapping between simulation data and real pipelines, supporting dynamic scenario switching. By designing various simulation scenarios (such as normal delivery, leakage, and corrosion), the performance of the digital twin model is tested in the simulation environment. Model outputs and performance indicators are recorded, and model parameters are adjusted based on the simulation results to optimize model performance.

[0112] S209. Use the K-nearest neighbor classifier and control chart method to regularly monitor the effectiveness of the digital twin model.

[0113] Specifically, the monitoring system extracts 10,000 sets of model output data under normal operating conditions from the simulation scenario, then injects 5% of calibrated abnormal data (such as pressure drops during simulated leaks and vibration spectrum shifts caused by corrosion). The algorithm parameters are optimized, and an improved DTW (Dynamic Time Warping) distance is used to target the periodic characteristics of pipeline data (such as vibration spectra): Where π is the alignment path, ∑(i,j)∈π is the sum of the distances of all matching points (i,j) on the path π, |x i -y j| is the absolute distance between the i-th point in sequence X and the j-th point in sequence Y. The K value is selected by the Elbow Method. In this embodiment, when K=5, the classification accuracy and computational efficiency are balanced.

[0114] Use the K-nearest neighbor algorithm to classify model outputs and identify abnormal patterns; use control charts to monitor model outputs and detect abnormal changes, regularly evaluate model performance to ensure its effectiveness, and monitor the effectiveness of digital twin models in real time to detect and fix problems in a timely manner.

[0115] In another embodiment, S104 specifically includes the following sub-steps:

[0116] S104.1. Build a complete edge computing environment.

[0117] Specifically, the monitoring system builds a complete edge computing environment, including hardware devices, applications, operating systems, and software stacks to achieve resource allocation and task scheduling; the monitoring system deploys edge computing devices (such as industrial computers, embedded devices, and smart gateways) to ensure that the devices have sufficient computing power, storage capacity, and network connectivity. The monitoring system installs lightweight operating systems (such as Linux distributions and real-time operating systems) on edge devices. The monitoring system deploys the necessary software stacks, including data processing frameworks (such as Apache Kafka and Flink), machine learning libraries (such as TensorFlow Lite and PyTorch), and communication protocols (such as MQTT and HTTP). Resource management tools (such as Kubernetes and Docker Swarm) are used to achieve dynamic resource allocation and task scheduling. The monitoring system provides a complete edge computing environment that supports the execution of data processing and analysis tasks, ensuring that edge devices can operate efficiently and meet real-time requirements.

[0118] S104.2. Use container technology to package the application and its dependencies into a container image.

[0119] Specifically, the monitoring system uses container technologies (such as Docker) to package applications and their dependencies into container images. These images are stored in image repositories (such as Docker Hub or private repositories) for easy deployment and updates. Deploying containerized applications on edge devices ensures rapid startup and operation. This improves application deployment efficiency and portability, simplifies dependency management, and ensures application consistency across different environments.

[0120] S104.3. Deploy edge computing services on the cloud platform and implement dynamic resource allocation and scheduling through virtualization technology.

[0121] Specifically, the monitoring system deploys edge computing services on a cloud platform using virtualization technologies (such as virtual machines and containers). This dynamically allocates computing and storage resources based on task requirements, ensuring efficient resource utilization. Task scheduling tools (such as Kubernetes and Apache Mesos) are used to dynamically schedule and load balance tasks. This supports large-scale and complex data processing requirements, improves resource utilization, and reduces cloud computing load.

[0122] S104.4. Deploy edge computing services at the edge of the network.

[0123] Specifically, the monitoring system deploys edge computing services at the edge of the network, utilizes network resources, expands coverage, and improves data transmission rate; the monitoring system deploys edge computing services at the edge of the network (such as base stations, routers), close to the data source, and utilizes network resources (such as 5G, edge gateways) to expand coverage and improve data transmission rate.

[0124] S104.5. Deploy micro data centers at the edge of the network to distribute data processing and storage to each node.

[0125] Specifically, the monitoring system deploys micro data centers at the edge of the network, equipped with computing, storage, and networking equipment. This disperses data processing and storage tasks across these micro data centers, enabling distributed processing. This improves data processing efficiency, reduces the load on centralized data centers, and enhances system reliability and fault tolerance.

[0126] S104.6. Combine cloud and edge resources to achieve data processing and analysis through a cloud-edge collaborative architecture.

[0127] Specifically, the cloud is responsible for collecting and preprocessing requests, performing large-scale data processing and model training, while the edge is responsible for real-time computing and response, handling tasks with high real-time requirements. This cloud-edge collaborative architecture enables dynamic allocation of data and tasks, optimizing resource utilization, and improving data processing efficiency, meeting the dual requirements of real-time performance and complexity, and optimizing the quality and personalization of AI-generated content.

[0128] S104.7. Optimize the model through pruning, quantization, and distillation techniques to adapt to the computing power resources of edge devices.

[0129] Specifically, model pruning removes redundant parameters from the model, reducing computational complexity; model quantization converts model parameters from floating-point numbers to low-precision values, reducing computational and storage requirements; and model distillation uses a large model to guide the training of smaller models, improving their performance. This helps optimize the model to adapt to the computing power of edge devices, improving operational efficiency and real-time performance.

[0130] S104.8. Schedule and allocate tasks based on their complexity, priority, and resource requirements.

[0131] Specifically, the monitoring system classifies tasks according to their complexity, priority, and resource requirements, and uses scheduling algorithms (such as priority scheduling and load balancing) to assign tasks to the nearest edge nodes, ensuring that the resources of the edge nodes are fully utilized and avoiding resource waste, which is conducive to improving task execution efficiency and ensuring that critical tasks are completed first.

[0132] In another embodiment, S105 specifically includes the following sub-steps:

[0133] S105.1. Extract key features based on the sensor information.

[0134] Specifically, the monitoring system uses statistical methods, signal processing techniques, or deep learning algorithms (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs)) to extract key features from sensor information. These features include pipeline pressure trends, temperature fluctuations, and flow anomalies. Extracting key features simplifies subsequent analysis and improves algorithm efficiency and accuracy. Key features can more intuitively reflect the operating status of the oil pipeline system and help identify potential problems.

[0135] S105.2. Use a machine learning algorithm to model the key features and predict the remaining useful life of the oil pipeline system.

[0136] Specifically, the monitoring system uses a bidirectional LSTM+Attention mechanism based on key features to capture long-term and short-term dependencies. Physical features are fused, and physical quantities such as corrosion rate are injected into the LSTM output layer via residual connections. The output layer is designed with a Gamma process regression layer to output the probability distribution of RUL. A Transformer encoder is used to process the vibration spectrogram of the key features (converting it into a Mel-like spectrum), and a fully connected network is used to process the static features of the key features. A FocalLoss loss function is used to address class imbalance.

[0137] Based on the characteristics of the data and the complexity of the problem, a machine learning algorithm is selected. The preprocessed data and extracted features are used to train the selected machine learning algorithm to establish a predictive model. The model's performance is evaluated through cross-validation, and necessary adjustments and optimizations are made based on the evaluation results. The predictive model can provide an estimate of the remaining useful life of the oil pipeline system, providing an important basis for maintenance planning and resource allocation. Continuous model optimization can improve prediction accuracy and reduce false positives and false negatives.

[0138] S105.3. Identify potential failures in the oil pipeline system by analyzing the output of the digital twin model.

[0139] Specifically, the monitoring system converts the LSTM model into a TensorRT engine, achieving inference latency of less than 30ms (NVIDIA Jetson AGX Xavier). It uses ONNX Runtime for cross-platform deployment (compatible with x86 and ARM architectures). Edge nodes upload feature statistics every 24 hours. The cloud retrains the model based on incremental data and pushes updated parameters via over-the-air (OTA) push. When the predicted RUL value falls below a threshold (e.g., 30 days), a high-precision fault classification model is triggered. The SHAP value is combined to interpret the contribution of key features and locate the root cause of the fault (e.g., an abnormal vibration frequency of 125Hz indicates bearing wear).

[0140] Based on the physical characteristics and operational data of the oil pipeline system, a high-precision digital twin model is constructed. This model is then used to simulate the system's operational status, analyzing the system's response under different operating conditions and fault conditions. The simulation results are then compared with real-time data to identify abnormal patterns and potential faults. The digital twin model can reflect the operational status of the oil pipeline system in real time and provide comprehensive fault detection capabilities. Through simulation analysis, potential faults can be detected in advance, preventing serious accidents.

[0141] S105.4. Combine the output results of the digital twin model to predict the remaining service life of the equipment.

[0142] Specifically, the monitoring system integrates the output of the digital twin model with the results of the machine learning prediction model, and uses ensemble learning methods (such as weighted averaging and voting) or deep learning models (such as deep neural networks) to conduct a comprehensive analysis of the two results. Based on this comprehensive analysis, a more accurate prediction of the equipment's remaining useful life is then provided. By combining the results of the digital twin model and the machine learning prediction model, the accuracy and reliability of life predictions can be further improved. This comprehensive analysis can provide stronger support for maintenance decisions and ensure the safe and stable operation of the oil pipeline system.

[0143] This embodiment discloses a method for combining edge computing with digital twins, which specifically includes the following steps:

[0144] S301. When edge computing resources are limited, lightweight modeling technology is used to build a simplified twin model that adapts to the edge computing environment.

[0145] Specifically, the monitoring system first identifies key components in the oil pipeline system, which are typically the main factors affecting system performance and safety. It then determines the key parameters and status indicators for these key components. The monitoring system uses lightweight modeling techniques, such as dimensionality reduction and feature selection, to construct simplified digital twin models adapted to edge computing environments. These models significantly reduce computational complexity and resource requirements while maintaining accuracy. The monitoring system deploys the constructed simplified twin models to edge devices, such as sensor nodes and edge servers, to acquire and process data in real time. The simplified twin models can reflect the status changes of various components in the oil pipeline system in real time and provide accurate monitoring information. Due to the lightweight model, the computational and resource consumption of edge devices is greatly reduced, improving the overall efficiency and reliability of the system. Edge computing enables the simplified twin models to reflect the status changes of various components in the oil pipeline system in real time, enabling precise monitoring and rapid response to complex systems.

[0146] S302: adopting a physical model based on a discrete element model and a data-driven model based on a deep Boltzmann machine.

[0147] Specifically, the monitoring system performs discretization modeling, discretizing the pipeline wall into polyhedral units (such as Voronoi meshes). The unit size is set according to the characteristic scale of corrosion (typically 0.1-1 mm). Boundary conditions are applied, such as internal pressure load (Pinternal = 8 MPa) and soil constraint force (based on the Mohr-Coulomb criterion). A deep Boltzmann machine (DEM) outputs the coordinates of stress concentration areas, which serve as the DBM's attention mask to focus on key regional features. The corrosion expansion path predicted by the DEM is converted into discrete elements, which are the temporal constraints of the DBM input features. Abnormal vibration modes identified by the DBM are fed back to the DEM to modify contact force model parameters, such as the local friction coefficient.

[0148] The monitoring system constructs both physical and data-driven models. A physical model based on a discrete element model is used to describe the physical characteristics of the oil pipeline system, while a data-driven model based on a deep Boltzmann machine is used to capture the system's dynamic behavior. The monitoring system fuses the physical and data-driven models through a multi-layer feedforward neural network to construct a multi-information fusion (MIF) model. This model combines the advantages of both models to improve the accuracy of predictions and diagnoses. Based on real-time data feedback, the monitoring system adaptively updates the MIF model to reflect system performance degradation. This adaptive update mechanism enables the model to continuously adapt to system changes, maintaining the accuracy of predictions and diagnoses.

[0149] S303. Perform real-scene 3D visualization based on the temporal GIS system and conduct integrated detection in combination with modern remote sensing technology.

[0150] Specifically, the monitoring system uses a temporal geographic information system (GIS) to construct a realistic three-dimensional model of the oil pipeline system. This model displays information such as the pipeline system's spatial distribution and topography. The monitoring system integrates modern remote sensing technologies, such as satellite remote sensing and drone inspections, to conduct integrated inspections of the oil pipeline system. Remote sensing technology provides high-resolution images and data, helping to identify potential safety hazards. The monitoring system combines remote sensing data with a temporal GIS system for visual analysis. This intuitive three-dimensional model and data analysis provide decision support for managers.

[0151] This embodiment discloses a method for predicting the potential losses and impacts caused by pipeline failures by building a mathematical model based on pipeline failure characteristics, which specifically includes the following steps:

[0152] S401. Establish a distribution function of pipeline failure occurrence probability.

[0153] Specifically, the monitoring system collects historical data on pipeline failures, including failure type, occurrence time, and failure intensity. It then performs statistical analysis on the data to determine the frequency and patterns of failures. Based on the statistical results, it selects an appropriate probability distribution function (such as exponential, normal, or Weibull) to describe the probability of pipeline failures. The distribution function is then fitted to the failure data to obtain the distribution function parameters. This accurately describes the probability distribution of pipeline failures, providing a foundation for subsequent risk assessment.

[0154] The monitoring system defines pipeline failure intensity, such as leakage volume and pressure loss, and determines a loss function. This function should be able to reflect the losses caused by the failure intensity to the system, environment, economy, etc. The risk value is calculated based on the failure probability and failure intensity combined with the loss function, thus realizing a quantitative assessment of the losses that may be caused by pipeline failures.

[0155] S402. Construct a fuzzy membership function for risk assessment.

[0156] Specifically, the monitoring system selects core risk factors, such as corrosion depth, vibration intensity, pressure fluctuations, temperature gradients, and acoustic emission energy. Each indicator requires a clear quantification method (e.g., corrosion depth expressed as a percentage of wall thickness loss). Each indicator is categorized into three fuzzy levels: low, medium, and high. For example, corrosion depths below 5% are considered low risk, 5%-15% are considered medium risk, and above 15% are considered high risk. The membership of each level is defined using trapezoidal or triangular functions. For example, a corrosion depth of 10% might belong to both the "medium risk" and "high risk" fuzzy sets, with memberships of 0.6 and 0.4, respectively. The Analytic Hierarchy Process (AHP) is used to assign weights to different rules, ensuring that key factors (such as corrosion depth) have a greater impact on the results. The fuzzy inference results are converted into a specific risk index. For example, the "center of gravity method" is used to calculate the weighted average of each risk level, ultimately outputting a risk value between 0 and 1 (below 0.3 is low risk, 0.3-0.7 is medium risk, and above 0.7 is high risk).

[0157] Fuzzy set theory is used to describe the uncertainty of pipeline failures and construct a fuzzy membership function for risk assessment. The sources of uncertainty in pipeline failures, such as environmental factors, material properties, and operating conditions, are identified. Fuzzy set theory is then used to construct a fuzzy membership function to describe the impact of each uncertainty factor on the probability and severity of failures. By incorporating uncertainty factors into the risk assessment model through fuzzy operations, the uncertainty issue in pipeline failure assessment is better addressed.

[0158] S403. Use a neural network to process nonlinear relationships in pipeline fault assessment.

[0159] Specifically, the monitoring system integrates sensor data (vibration, temperature, pressure, etc.) and maintenance records (corrosion measurements, fault type labels) and generates time series features, such as the root mean square (RMS) of vibration and the rate of change of pressure over the past hour; it also fuses multi-sensor data, such as the mutual information between vibration and temperature (which measures the dynamic correlation between the two). Designing the neural network architecture:

[0160] Input layer: The number of nodes is consistent with the feature dimension (e.g., 7 input nodes correspond to 7 sensor features).

[0161] Hidden layer: The first layer has 64 nodes and uses the ReLU activation function to capture nonlinear relationships. The second layer has 32 nodes and uses the SELU activation function to improve self-normalization capabilities.

[0162] Output layer: 3 nodes (corresponding to three types of fault probabilities), outputting the classification results through the Softmax function.

[0163] Split the training and validation sets chronologically (e.g., 80% for training, 20% for validation) to prevent future data leakage. Add a Dropout layer (randomly drop 20% of nodes); use early stopping (terminating training if validation loss does not improve after 10 consecutive epochs). Use the Adam optimizer with an initial learning rate of 0.001 and a gradual decay. Output the fault type (e.g., corrosion, leakage, weld cracking) and its confidence level; use SHAP values (SHapleyAdditive exPlanations) to explain the contribution of key features. For example, an anomaly in the 125Hz vibration band may indicate bearing wear.

[0164] By collecting multimodal data related to pipeline failures, such as sensor data and environmental factor data, a neural network architecture (such as a fully connected neural network or a convolutional neural network) is selected and trained based on the data characteristics. The trained neural network model is then used to assess and predict pipeline failures. This allows for the processing of complex nonlinear relationships, improving the accuracy and precision of pipeline failure assessments.

[0165] S404: Query historical pipeline fault data from a preset database.

[0166] Specifically, users pre-establish a pipeline fault history database to store fault data and related information, and design a database query interface to facilitate user query of historical data based on needs. This enables rapid query and access to historical pipeline fault data.

[0167] S405. Use frequency analysis to estimate the probability of pipeline failure within a specific time.

[0168] Specifically, the monitoring system uses frequency analysis based on historical pipeline failure data to estimate the probability of pipeline failure occurring within a specific time period, and then establishes a pipeline failure probability distribution model; based on historical pipeline failure data, the number of occurrences of each type of failure within a specific time period is counted, and the frequency of occurrence of each type of failure is calculated, that is, the ratio of the number of failures to the total time or total number of times. Using frequency analysis, the probability of pipeline failure occurring within a specific time period is estimated, providing an intuitive estimate of the probability of pipeline failure.

[0169] S406: Use a multiple linear regression model to analyze the relationship between the sensor information and the probability of pipeline failure.

[0170] Specifically, the monitoring system uses stepwise regression to screen significant variables (such as corrosion depth, vibration RMS, and pressure fluctuations), calculates the variance inflation factor (VIF), and eliminates variables with a VIF greater than 5 (for example, selecting both vibration RMS and vibration peak-to-peak values simultaneously can lead to collinearity). A logistic regression model is established, with sensor features as input and the probability of failure (between 0 and 1) as output. Feature combinations, such as "vibration intensity × corrosion rate," are constructed to capture synergistic effects. A significance test is performed, using p-values (less than 0.05 is significant) and confidence intervals to verify variable importance. The failure probability corresponding to the current sensor data is calculated in real time. The model is regularly refitted with new data to adapt to changing operating conditions. A multi-model collaborative decision-making process is implemented: the first-level assessment uses fuzzy logic to calculate the risk index in real time. If it exceeds a threshold (such as 0.7), a detailed neural network diagnosis is triggered. The second-level diagnosis uses the neural network to output the fault type and confidence level, combining this with the failure probability predicted by the regression model to generate a comprehensive report. Actual maintenance results are fed back to the system to update the fuzzy rule base and model parameters.

[0171] Based on the sensor data, the dependent variable (pipeline failure probability) and the independent variable (sensor data) were determined. A multivariate linear regression model was then established to analyze the impact of the independent variables on the dependent variable. The model was then tested and optimized to ensure its accuracy and reliability. This revealed the inherent connection between sensor data and pipeline failure probability, providing a theoretical basis for sensor data-based pipeline failure prediction.

[0172] This embodiment discloses a method for setting and verifying mathematical model parameters in pipeline fault analysis, which specifically includes the following steps:

[0173] S501. Determine the parameters that need to be set according to the specific requirements of pipeline fault analysis.

[0174] Specifically, the monitoring system first defines the objective of pipeline failure analysis, such as predicting failure rate, identifying fault type, or locating fault location. Based on this objective, it then analyzes various factors that may influence pipeline failure, such as pipe diameter, material, age, internal pressure, temperature, and external load. Parameters with significant impact on pipeline failure are selected from these factors and used as input variables in the mathematical model. This ensures that the input variables of the mathematical model are closely aligned with the objective of pipeline failure analysis, improving the model's predictive accuracy and practicality.

[0175] S502: Using a Bayesian model selection method to determine the relationship between pipeline faults and different mathematical model parameters, and defining the mathematical model space dimension and initial probability distribution.

[0176] Specifically, the monitoring system identifies candidate models and, based on the pipeline failure mechanism, lists possible mathematical models (such as exponential corrosion models, stress fatigue models, and leakage diffusion models). Each model corresponds to a set of parameters to be estimated (such as corrosion rate, crack growth coefficient, and leakage aperture threshold). An independent parameter space is allocated to each model. For example, the exponential corrosion model has two parameter dimensions (corrosion coefficient and environmental sensitivity factor), while the stress fatigue model has three parameter dimensions (initial defect size, stress amplitude, and material toughness).

[0177] Integrate sensor time series data (vibration, pressure, temperature) and fault records (such as measured corrosion depth and leak event time) and select a likelihood function: Gaussian likelihood for continuous data (such as corrosion rate) and Poisson or Bernoulli likelihood for discrete events (such as leak occurrence). Each model's ability to explain the data is assessed using marginal likelihood (model evidence). For complex models, Bayesian Information Criterion (BIC) or variational inference are used as approximations instead of exact calculations.

[0178] Using Bayesian theorem, based on prior knowledge and historical data, we calculated the posterior probability between pipeline failures and different mathematical model parameters. We also defined the spatial dimension of the mathematical model, namely the number and range of input variables. Based on historical data and expert experience, we set an initial probability distribution, which served as the basis for subsequent Gibbs sampling. This Bayesian model selection method allows for a flexible description of the relationship between pipeline failures and different parameters. Setting this initial probability distribution provides the foundation for subsequent sampling and parameter estimation.

[0179] S503: Use Gibbs sampling to perform posterior estimation of parameters, through multiple iterations and discarding some samples.

[0180] Specifically, the monitoring system randomly extracts initial parameter values from the prior distribution (e.g., the initial value of the corrosion coefficient = 0.3 mm / year). Multiple Markov chains (usually 3 to 5) are run in parallel to detect convergence consistency. Other parameters are fixed and individual parameters are updated sequentially: for example, the corrosion coefficient is updated first, and the conditional distribution is calculated based on the current stress amplitude and observed data; then the stress amplitude is updated and resampled based on the new corrosion coefficient. Variables are selected for the discrete model, and the posterior probability of each model is calculated and sampled. Continuous parameters use the Metropolis-Hastings algorithm or adaptive sampling; discrete model variables are directly sampled through the category distribution. The first 20% of iterative samples are discarded (e.g., for a total of 10,000 iterations, the first 2,000 are not retained) to eliminate the influence of the initial value.

[0181] Based on the defined mathematical model space dimensions and initial probability distribution, the Gibbs sampling algorithm is used for multiple iterations. In each iteration, the value of each parameter is updated based on the current parameter state and the conditional probability distribution of other parameters. By discarding some samples from the early stages of the iteration, the autocorrelation of the sampling sequence is reduced, thereby improving the accuracy of parameter estimation. The Gibbs sampling algorithm simplifies sampling of high-dimensional continuous probability distributions, improving the efficiency of parameter estimation. By repeating multiple iterations and discarding some samples, the posterior distribution of the parameters can be more accurately estimated.

[0182] S504: Calculate the mean and standard deviation of the posterior distribution to determine the acceptability of the parameters.

[0183] Specifically, the monitoring system calculates the mean and standard deviation of the posterior distribution based on parameter samples obtained through Gibbs sampling. Based on these values, the system assesses the stability and reliability of the parameters. Based on actual needs, the system sets an acceptable range for the parameters and determines whether the parameters are within this acceptable range. By calculating the mean and standard deviation of the posterior distribution, the system can quantify the uncertainty and stability of the parameters, providing a basis for parameter optimization and adjustment.

[0184] S505. Use historical pipeline failure data to verify the mathematical model.

[0185] Specifically, the monitoring system collects historical pipeline fault data, including information such as fault time, fault type, fault location, and related parameters. This historical data is then fed into a mathematical model to predict faults. The predicted results are then compared with actual historical data to assess the accuracy and reliability of the mathematical model. Verification with historical data ensures that the mathematical model's predictions match actual data, improving the accuracy and credibility of the mathematical model in practical applications.

[0186] S506: Compare the simulation results with the actual accident records and perform statistical analysis.

[0187] Specifically, the monitoring system collects actual accident records, including information such as fault time, fault type, and fault location. Using mathematical models, the system performs simulations to predict pipeline failures. The simulation results are then compared with actual accident records, and statistical analysis is performed to assess the consistency between the mathematical model output and the actual data. By comparing simulation results with actual accident records, the accuracy and reliability of the mathematical model can be further verified.

[0188] Based on the above method, the embodiment of the present application also discloses a system for monitoring and evaluating the quality of an oil pipeline based on data processing. A system for monitoring and evaluating the quality of an oil pipeline based on data processing includes:

[0189] The perception layer is used to perceive physical entities and their operating environment in real time, and collect status data of physical entities through sensors and IoT devices;

[0190] Data layer, used for data collection, storage, processing and transmission;

[0191] The modeling layer is used to transform the data of physical entities into virtual models through mechanism modeling and data-driven modeling;

[0192] The interactive layer is used to provide user interface, visualization, and report output functions;

[0193] The decision support layer is used to realize automated decision support of the digital twin system through intelligent optimization algorithms and data mining technology. It uses historical data and real-time data to analyze, predict future trends, and provide support for decision-making;

[0194] Security assurance layer, including data encryption, access control, security auditing, and disaster recovery measures, to ensure system security and stability;

[0195] Platform software and mechanism analysis layer, including cloud computing platform, edge computing, stream computing, and in-memory computing technologies, to improve data processing efficiency and real-time performance;

[0196] The multi-source data fusion layer involves the integration of multiple types of data, including environmental data, maintenance data, and operation data, to ensure the comprehensiveness and accuracy of the data.

[0197] An embodiment of the present application further discloses an intelligent terminal comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed by the above-mentioned data processing-based oil pipeline quality monitoring and evaluation method.

[0198] The present application also discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program capable of being loaded by a processor and executing the aforementioned data processing-based oil pipeline quality monitoring and assessment method. Examples of the computer-readable storage medium include various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0199] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the embodiments described are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also fall within the scope of protection of the present invention.

Claims

1. A method for monitoring and evaluating oil pipeline quality based on data processing, characterized in that: The following steps are involved: Acquiring sensor information, including visual, sound, temperature, and vibration data; Obtain application scenario information; Through early fusion, mid-term fusion and late fusion strategies, the fusion method is selected in combination with application scenario information; Using edge computing methods, computing and storage resources are deployed on edge nodes close to data sources. These edge nodes are responsible for preliminary data processing and analysis, reducing data transmission delays. Analyzing the sensor information using deep learning and machine learning algorithms to identify abnormal patterns and potential faults; The edge computing is combined with cloud computing to form a cloud-edge collaborative architecture, which is used to perform complex data processing and model training.

2. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: The edge computing is combined with digital twins, specifically including: When edge computing resources are limited, lightweight modeling technology is used to build a simplified twin model that adapts to the edge computing environment. This edge computing enables the simplified twin model to reflect the status changes of each component of the oil pipeline system in real time, achieving accurate monitoring and rapid response to complex systems. Adopting a physical model based on discrete element model and a data-driven model based on deep Boltzmann machine, a MIF model is constructed through a multi-layer feedforward neural network to achieve adaptive update of oil pipeline system performance degradation; Real-scene 3D visualization is performed based on the temporal GIS system, and integrated detection is performed in combination with modern remote sensing technology.

3. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: After the step of obtaining sensor information, the method further includes: Clean and preprocess the data; Performing HIS transformation and wavelet decomposition on the vibration data and temperature data to extract joint features; Query historical data and key tasks from the preset database; Design priority queues to ensure that critical tasks occupy more than 80% of computing resources; Building a digital twin model based on the historical data and machine learning algorithms, suitable for complex and difficult-to-model oil pipeline systems; Verify the validity of the unit-level model of the digital twin model and ensure the high fidelity of the basic unit-level model; On the basis of ensuring the validity of the unit-level model of the digital twin model, further verify the assembled or fused digital twin model; Through simulation experiments, the performance of the digital twin model in different scenarios is tested and optimized based on the simulation results; The effectiveness of the digital twin model is regularly monitored using a K-nearest neighbor classifier and control chart methods.

4. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: The step of using edge computing methods to deploy computing and storage resources on edge nodes close to data sources specifically includes: Build a complete edge computing environment, including hardware devices, applications, operating systems, and software stacks, to achieve resource allocation and task scheduling; Using container technology to package the application and its dependencies into a container image; Deploy edge computing services on cloud platforms and implement dynamic resource allocation and scheduling through virtualization technology to meet large-scale and complex data processing needs; Deploy edge computing services at the edge of the network to utilize network resources, expand coverage, and increase data transmission rates; Deploy micro data centers at the edge of the network to disperse data processing and storage to various nodes, enabling distributed data processing. Combining cloud and edge resources to enable data processing and analysis through a cloud-edge collaborative architecture, with the cloud collecting and preprocessing requests and the edge performing real-time computation and response, optimizing the quality and personalization of AI-generated content. Optimize the model through pruning, quantization, and distillation techniques to adapt to the computing power resources of edge devices; Scheduling and allocation are performed based on the complexity, priority, and resource requirements of the task, and the task is assigned to the nearest edge node for processing to ensure that the resources of the edge node are fully utilized.

5. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: The step of analyzing the sensor information using deep learning and machine learning algorithms to identify abnormal patterns and potential faults also includes: extracting key features based on the sensor information; Modeling the key features using a machine learning algorithm to predict the remaining useful life of the oil pipeline system; Identify potential failures in the oil pipeline system by analyzing the output of the digital twin model; Combined with the output of the digital twin model, the remaining service life of the equipment is predicted.

6. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: Based on the characteristics of pipeline failures, a mathematical model is constructed to predict the losses and impacts that may be caused by pipeline failures, including: Establish a distribution function for the probability of pipeline failure, combine the pipeline failure intensity and loss function, and calculate the risk value; Fuzzy set theory is used to describe the uncertainty of pipeline failures and fuzzy membership functions are constructed for risk assessment. Using neural networks to handle nonlinear relationships in pipeline fault assessment; Query historical pipeline failure data from the preset database; Based on historical pipeline failure data, the frequency analysis method is used to estimate the probability of pipeline failure within a specific time period, and then a pipeline failure probability distribution model is established; A multiple linear regression model is used to analyze the relationship between the sensor information and the probability of pipeline failure.

7. The oil pipeline quality monitoring and evaluation method based on data processing according to claim 1 is characterized in that: Setting and validating parameters for mathematical models used in pipeline failure analysis, including: Determine the parameters that need to be set based on the specific needs of pipeline fault analysis; The Bayesian model selection method is used to determine the relationship between pipeline faults and different mathematical model parameters, and to define the mathematical model space dimension and initial probability distribution; Use Gibbs sampling to perform posterior estimation of parameters, and reduce the autocorrelation of the sampling sequence by multiple iterations and discarding some samples; Calculate the mean and standard deviation of the posterior distribution to determine the acceptability of the parameters; Use historical pipeline failure data to validate the mathematical model to ensure that the model's predictions are consistent with actual data; Compare the simulation results with the actual accident records and conduct statistical analysis to ensure that the output results of the mathematical model are consistent with the actual data.

8. A data processing-based oil pipeline quality monitoring and evaluation system, characterized in that: include: The perception layer is used to perceive physical entities and their operating environment in real time, and collect status data of physical entities through sensors and IoT devices; Data layer, used for data collection, storage, processing and transmission; The modeling layer is used to transform the data of physical entities into virtual models through mechanism modeling and data-driven modeling; The interactive layer is used to provide user interface, visualization, and report output functions; The decision support layer is used to realize automated decision support of the digital twin system through intelligent optimization algorithms and data mining technology. It uses historical data and real-time data to analyze, predict future trends, and provide support for decision-making; Security assurance layer, including data encryption, access control, security auditing, and disaster recovery measures, to ensure system security and stability; Platform software and mechanism analysis layer, including cloud computing platform, edge computing, stream computing, and in-memory computing technologies, to improve data processing efficiency and real-time performance; The multi-source data fusion layer involves the integration of multiple types of data, including environmental data, maintenance data, and operation data, to ensure the comprehensiveness and accuracy of the data.

Images