Intelligent oil pipe inspection robot system and method based on multi-mode machine learning
By integrating lightweight detection models and multimodal fusion modules with multimodal machine learning technology, combined with composite support mechanisms and intelligent decision engines, the perception and real-time issues in oil and gas pipeline inspections are solved, enabling efficient and accurate detection and prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-04-03
AI Technical Summary
Existing oil and gas pipeline inspection technologies suffer from insufficient sensing capabilities, algorithm performance defects, real-time limitations, and a lack of predictive capabilities, resulting in low detection efficiency, high false negative rates, and failure to meet API 1160 specification requirements.
An intelligent pipeline inspection robot system based on multimodal machine learning is adopted, which integrates a lightweight detection model module, a multimodal dynamic fusion module, a federated learning incremental module, a pipeline adaptive robot module, and an intelligent decision engine module. Through technologies such as HetConv hybrid convolution, ShuffleAttention, illumination adaptive weighted fusion, edge knowledge distillation, composite support mechanism, and intelligent decision engine, efficient detection and prediction are achieved.
It improves the identification rate of minute corrosion points to 92.1%, reduces edge inference latency to ≤80ms, lowers the false negative rate and maintenance costs, and meets the requirements of API 1160 specification.
Smart Images
Figure CN121786449A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of oil and gas pipeline inspection technology, specifically to an intelligent oil pipeline inspection robot system and method based on multimodal machine learning. Background Technology
[0002] Safety inspection of oil and gas pipelines is a core aspect of energy infrastructure operation and maintenance, but current mainstream technologies have significant shortcomings. Traditional manual inspections rely on visual inspection and handheld devices (such as ultrasonic thickness gauges), requiring partial pipeline shutdowns and exhibiting low inspection efficiency, with a daily coverage length of less than 500 meters. They also struggle to reach high-altitude and underground pipeline sections, and the rate of missed detection for minute corrosion points (<2mm) exceeds 28%. While existing pipeline inspection robots improve coverage, they still face key technological bottlenecks:
[0003] 1. Insufficient sensing capability: The single-mode visible light camera produces blurry images in environments with oil stains or sudden changes in illumination (0.01-200 lux), resulting in a false alarm rate of >31.6%;
[0004] 2. Algorithm performance defects: General target detection models (such as YOLOv8) are not optimized for small target detection, and the recognition rate for corrosion points smaller than 2mm is only 65%;
[0005] 3. Real-time limitations: Edge devices have limited computing power (≤5TOPS), and high-precision model inference latency >200ms, which cannot meet the 100ms-level real-time response requirements;
[0006] 4. Lack of predictive capability: There is a lack of corrosion evolution time series modeling, the life prediction error is >15%, the maintenance plan relies on human experience, and the economic losses from downtime are not quantified.
[0007] According to the API 1160 specification, an ideal inspection system must simultaneously achieve: a defect recognition rate of ≥95%, daily coverage of ≥3km, latency of ≤100ms, and corrosion prediction error of <5%. Current technologies do not yet meet these requirements. There is an urgent need to break through key technologies such as multimodal perception fusion, lightweight edge intelligence, and corrosion trend prediction to address the three major industry pain points of poor environmental adaptability, insufficient real-time performance, and crude decision-making. Summary of the Invention
[0008] The purpose of this invention is to provide an intelligent pipeline inspection robot system and method based on multimodal machine learning to solve the problems mentioned in the background art.
[0009] To achieve the above objectives, the present invention provides the following technical solution: an intelligent pipeline inspection robot system based on multimodal machine learning, comprising a lightweight detection model module, a multimodal dynamic fusion module, a federated learning incremental module, a pipeline adaptive robot module, and an intelligent decision engine module;
[0010] The lightweight detection model module adopts HetConv hybrid convolution and ShuffleAttention mechanism, and designs Inner-SIoU loss function to achieve a 2mm erosion point recognition rate of ≥92.1% and an edge inference latency of ≤80ms;
[0011] The multimodal dynamic fusion module achieves multi-source data fusion by adaptively weighting visible light and infrared features through illumination, and by coordinating with laser data.
[0012] The federated learning incremental module adopts a two-level update architecture of edge knowledge distillation and cloud-based FedProx aggregation.
[0013] The pipeline adaptive robot module is equipped with a helical spring-torsion spring composite support mechanism and achieves path planning based on DDPG reinforcement learning;
[0014] The intelligent decision engine module integrates an LSTM corrosion prediction model and a genetic algorithm maintenance optimization model.
[0015] Preferably, the lightweight detection model module specifically includes: adopting a HetConv hybrid convolutional structure, connecting 3×3 and 1×1 convolutions in parallel, reducing computation by ≥9%, and FLOPs≤4.2G; integrating a ShuffleAttention mechanism to enhance the ability to extract oil pollution interference features through channel grouping and spatial recalibration; and designing an Inner-SIoU loss function to focus on gradient updates for small targets.
[0016] Preferably, the multimodal dynamic fusion module specifically includes:
[0017] The feature-level fusion unit dynamically adjusts the weights of visible light and infrared features using an illumination-adaptive weighted algorithm. The weight value α is dynamically adjusted according to the illumination intensity lux. When lux > 100, α = 0.8; otherwise, α = 0.3. The fusion formula is as follows:
[0018] fusion_feat=α*vis_feat+(1-α)*ir_feat;
[0019] The multi-source data collaboration unit uses ResNet18-FPN to extract visible light texture features, UNet to segment infrared temperature anomaly areas, and a leak warning is triggered when the temperature difference is greater than 5℃. The LiDAR quantifies the depth of corrosion pits with an accuracy of ±0.1mm.
[0020] Preferably, the federated learning incremental module specifically includes: using knowledge distillation technology to compress the model at the edge to adapt to the RK3588 NPU; using the FedProx algorithm in the cloud to aggregate gradients from multiple nodes to solve the problem of non-independent and identically distributed data; and improving the weekly recognition accuracy of the model through a two-level update architecture.
[0021] Preferably, the pipeline adaptive robot module specifically includes:
[0022] The mechanical structure unit adopts a composite support mechanism of helical springs and torsion springs to achieve dynamic adjustment of contact pressure: 5-15N, and the obstacle-crossing ability covers 15mm weld seams and corrosion protrusions.
[0023] The navigation decision unit, based on DDPG reinforcement learning, constructs a path planning model, and the reward function is defined as:
[0024] reward = 0.6 information entropy + 0.3 defect density - 0.1 * energy consumption;
[0025] In the intelligent decision engine module, the LSTM time series model takes historical corrosion data and environmental parameters (temperature / humidity / medium) as input and outputs a remaining life assessment error of <3.2%; the genetic algorithm optimizes the maintenance strategy with the objective function of minimizing maintenance costs and downtime losses, and the constraint condition is that the risk level is greater than the threshold.
[0026] A method for intelligent pipeline inspection robots based on multimodal machine learning includes the following steps:
[0027] A lightweight detection model is constructed, which achieves efficient feature extraction through HetConv hybrid convolution and ShuffleAttention mechanism, and optimizes small target detection using Inner-SIoU loss function;
[0028] A multimodal dynamic fusion method is implemented, which fuses visible light and infrared features based on an illumination adaptive weighting strategy, and coordinates with laser data to achieve multi-source information fusion;
[0029] Establish a federated learning incremental system to achieve dynamic model updates through edge knowledge distillation and cloud-based FedProx aggregation;
[0030] Design a pipeline adaptive robot, and use a composite support mechanism and DDPG reinforcement learning to achieve contact pressure regulation and path planning;
[0031] Develop an intelligent decision engine that integrates an LSTM corrosion prediction model with a genetic algorithm maintenance optimization model to complete the tubing condition assessment and maintenance strategy formulation.
[0032] Preferably, the steps for constructing a lightweight detection model specifically include: adopting a HetConv hybrid convolutional structure, connecting 3×3 and 1×1 convolutions in parallel to reduce computation by ≥9% and achieve FLOPs≤4.2G; integrating a ShuffleAttention mechanism to enhance the feature extraction capability of oil stain interference through channel grouping and spatial recalibration; designing an Inner-SIoU loss function to focus on gradient updates for small targets, thereby improving the mAP@0.5 index by ≥4.6% in low-light environments, achieving a 2mm corrosion point recognition rate of ≥92.1% and an edge inference latency of ≤80ms.
[0033] Preferably, the steps for implementing the multimodal dynamic fusion method specifically include:
[0034] In the feature-level fusion step, the weights of visible light and infrared features are dynamically adjusted using an illumination-adaptive weighted algorithm. The weight value α is dynamically set according to the illumination intensity lux. When lux > 100, α = 0.8; otherwise, α = 0.3. The fusion formula is as follows:
[0035] fusion_feat=α*vis_feat+(1-α)*ir_feat;
[0036] The multi-source data collaboration process employs ResNet18-FPN to extract visible light texture features, UNet to segment infrared temperature anomaly areas: a temperature difference > 5℃ triggers a leak warning, and lidar quantifies the depth of corrosion pits with an accuracy of ±0.1mm.
[0037] Preferably, the steps for establishing a federated learning incremental system specifically include: an edge-side knowledge distillation step to compress the model to fit the RK3588 NPU; a cloud-based FedProx aggregation step to aggregate gradients from multiple nodes to address the problem of non-independent and identically distributed data; and an improvement of the model's weekly recognition accuracy by ≥0.8% through a two-level update architecture.
[0038] Preferably, the steps for designing a pipeline adaptive robot and developing an intelligent decision engine specifically include:
[0039] The mechanical structure is innovative, employing a composite support mechanism of helical springs and torsion springs to achieve dynamic adjustment of contact pressure and enable it to overcome obstacles such as 15mm welds and corrosion protrusions.
[0040] The navigation decision-making steps are based on a path planning model constructed using DDPG reinforcement learning, with the reward function defined as follows:
[0041] reward = 0.6 information entropy + 0.3 defect density - 0.1 * energy consumption;
[0042] The corrosion prediction step involves inputting historical corrosion data and environmental parameters (temperature, humidity, and medium) into an LSTM time series model, and outputting a remaining lifetime assessment error of <3.2%.
[0043] The maintenance optimization steps employ a genetic algorithm with the objective function of minimizing maintenance costs and downtime losses, under the constraint that the risk level is greater than the threshold.
[0044] Compared with the prior art, the beneficial effects of the present invention are:
[0045] This invention proposes an intelligent pipeline inspection robot system and method based on multimodal machine learning. It integrates HetConv hybrid convolution and ShufFleAttention modules, improving the recognition rate of minute corrosion points (<2mm) from 65% to 92.1% under oil pollution interference, with edge inference latency ≤80ms. It solves the problem of missed detection in low-light environments (0.01 lux) by fusing visible light texture features and infrared temperature anomaly data through illumination-adaptive weight α. An edge-side knowledge distillation and compression model, along with cloud-based aggregation of multi-robot data to update the global defect detection model, improves weekly accuracy by 0.8%. A composite spring support mechanism dynamically adjusts the contact pressure (5-15N) to adapt to pipe diameters from DN80 to DN450, with an obstacle clearance height of up to 15mm. An LSTM model predicts the corrosion rate (error <3.2%), and combined with a genetic algorithm to generate maintenance plans, reducing maintenance costs by 60%. Attached Figure Description
[0046] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of the present invention clear and complete, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only some, not all, embodiments of the present invention, and are merely illustrative of the embodiments of the present invention. They are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0048] Example 1: This invention provides a technical solution: an intelligent pipeline inspection robot system based on multimodal machine learning, comprising the following core modules:
[0049] 1. Lightweight detection model (HSS-YOLO)
[0050] Structural innovation: HetConv hybrid convolution (3×3 and 1×1 in parallel) is used to replace standard convolution, reducing computation by 9% (FLOPs≤4.2G); ShuffleAttention mechanism is integrated to enhance the extraction of oil pollution interference features through channel grouping and spatial recalibration.
[0051] Loss function optimization: Inner-SIoU loss function was designed to focus on gradient updates for small targets, improving mAP@0.5 by 4.6% in low-light environments; Performance metrics: 92.1% recognition rate for 2mm erosion points, and edge inference latency ≤80ms.
[0052] 2. Multimodal dynamic fusion method
[0053] Feature-level fusion:
[0054]
[0055] Multi-source data collaboration:
[0056] Visible light: ResNet18-FPN extracts texture features
[0057] Infrared: UNet segments temperature anomaly areas (temperature difference > 5℃ triggers leak warning)
[0058] Laser: Quantifying the depth of corrosion pits (accuracy ±0.1mm)
[0059] 3. Federated Learning Incremental System
[0060] Two-level update architecture:
[0061]
[0062] Evolutionary effect: The model's weekly average recognition accuracy improved by 0.8%.
[0063] 4. Pipeline Adaptive Robot
[0064] Mechanical Innovation: Composite support mechanism of helical spring + torsion spring, with dynamic adjustment of contact pressure (5-15N); Obstacle Crossing Ability: 15mm weld seam / corrosion protrusion; Navigation Decision: Path planning based on DDPG reinforcement learning, reward function:
[0065] reward = 0.6 * information entropy + 0.3 * defect density - 0.1 * energy consumption.
[0066] 5. Intelligent Decision Engine
[0067] Corrosion prediction:
[0068] Input to the LSTM time series model: historical corrosion data + environmental parameters (temperature / humidity / medium).
[0069] Output: Remaining lifetime assessment (error < 3.2%)
[0070] Maintenance and optimization:
[0071] Genetic Algorithm Multi-Objective Optimization:
[0072] #MATLAB#
[0073] min[maintenance costs, downtime losses]
[0074] The risk level is above the threshold.
[0075] Example 2, based on Example 1, proposes a method for intelligent pipeline inspection robot based on multimodal machine learning, specifically for intelligent inspection of crude oil transportation pipelines:
[0076] Scene parameters:
[0077] Pipeline specifications: DN300 carbon steel pipeline, length 5km, medium temperature 60℃
[0078] Environmental conditions: Oil adhesion level III (ISO 8501-1), light intensity fluctuating from 5 to 150 lux.
[0079] 1. Hardware Deployment
[0080]
[0081]
[0082] 2. Defect Detection Process
[0083] Step 1: Data Acquisition and Dynamic Cleaning
[0084] The robot travels to pipe section L2-3 (coordinate x=125m), detects oil stains obstructing the lens (transmittance <70%), triggers the micro motor to drive the scraper (speed 200rpm), and after cleaning, the transmittance is restored to 92%.
[0085] Step 2: Multimodal data fusion analysis
[0086] #python#
[0087] #Dynamic weight calculation (current light intensity = 8 lux)
[0088] When α = 0.3#, the infrared weighting is higher when the illumination is <10 lux.
[0089] vis_feat = ResNet18_FPN(visible_img) # Extract texture features
[0090] ir_mask = UNet(ir_img) # Segment the temperature anomaly region
[0091] fusion_feat=α*vis_feat+(1-α)*ir_mask #Weighted fusion output: Corrosion point detected (confidence 96.2%), coordinates (x,y)=(312px,408px) Step 3: 3D quantization and classification
[0092]
[0093] 3. Intelligent Decision-Making and Response
[0094] Decision chain logic:
[0095] Digital twin platform operation: Historical data retrieval: Corrosion depth at this point increased by 0.4 mm within 3 months; LSTM prediction: Remaining lifetime = 7.3 months (confidence interval ± 0.8 months);
[0096] Genetic Algorithm Optimization:
[0097] Option 1: Partial repairs after 3 months → Downtime loss of ¥800,000
[0098] Option 2: Pressurized plugging by the end of this month → Repair cost: ¥350,000
[0099] Output the optimal solution: Solution 2 (lowest overall cost).
[0100] 4. Federated learning model update
[0101] Edge end: Distillation-compression HSS-YOLO model (parameter count from 14.7M → 4.2M)
[0102] Cloud-based: Aggregate gradient data from 10 robots and update the global model.
[0103] Evolution effect:
[0104]
[0105] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A pipeline intelligent inspection robot system based on multimodal machine learning, characterized in that: It includes a lightweight detection model module, a multimodal dynamic fusion module, a federated learning incremental module, a pipeline adaptive robot module, and an intelligent decision engine module; The lightweight detection model module adopts HetConv hybrid convolution and ShuffleAttention mechanism, and designs Inner-SIoU loss function to achieve a 2mm erosion point recognition rate of ≥92.1% and an edge inference latency of ≤80ms; The multimodal dynamic fusion module achieves multi-source data fusion by adaptively weighting visible light and infrared features through illumination, and by coordinating with laser data. The federated learning incremental module adopts a two-level update architecture of edge knowledge distillation and cloud-based FedProx aggregation. The pipeline adaptive robot module is equipped with a helical spring-torsion spring composite support mechanism and achieves path planning based on DDPG reinforcement learning; The intelligent decision engine module integrates an LSTM corrosion prediction model and a genetic algorithm maintenance optimization model.
2. The intelligent pipeline inspection robot system based on multimodal machine learning according to claim 1, characterized in that: The lightweight detection model module specifically includes: adopting a HetConv hybrid convolutional structure, which connects 3×3 and 1×1 convolutions in parallel, reducing the computational cost by ≥9% and FLOPs≤4.2G; integrating the ShuffleAttention mechanism, which enhances the ability to extract oil pollution interference features through channel grouping and spatial recalibration; and designing an Inner-SIoU loss function to focus on gradient updates for small targets.
3. The intelligent pipeline inspection robot system based on multimodal machine learning according to claim 2, characterized in that: The multimodal dynamic fusion module specifically includes: The feature-level fusion unit dynamically adjusts the weights of visible light and infrared features using an illumination-adaptive weighted algorithm. The weight value α is dynamically adjusted according to the illumination intensity lux. When lux > 100, α = 0.8; otherwise, α = 0.
3. The fusion formula is as follows: fusion_feat=α*vis_feat+(1-α)*ir_feat; The multi-source data collaboration unit uses ResNet18-FPN to extract visible light texture features, UNet to segment infrared temperature anomaly areas, and a leak warning is triggered when the temperature difference is greater than 5℃. The LiDAR quantifies the depth of corrosion pits with an accuracy of ±0.1mm.
4. The intelligent pipeline inspection robot system based on multimodal machine learning according to claim 3, characterized in that: The federated learning incremental module specifically includes: using knowledge distillation technology to compress the model at the edge to adapt to the RK3588 NPU; using the FedProx algorithm in the cloud to aggregate gradients from multiple nodes to solve the problem of non-independent and identically distributed data; and improving the weekly recognition accuracy of the model through a two-level update architecture.
5. The intelligent pipeline inspection robot system based on multimodal machine learning according to claim 4, characterized in that: The pipeline adaptive robot module specifically includes: The mechanical structure unit adopts a composite support mechanism of helical springs and torsion springs to achieve dynamic adjustment of contact pressure: 5-15N, and the obstacle-crossing ability covers 15mm weld seams and corrosion protrusions. The navigation decision unit, based on DDPG reinforcement learning, constructs a path planning model, and the reward function is defined as: reward = 0.6 information entropy + 0.3 defect density - 0.1 * energy consumption; In the intelligent decision engine module, the LSTM time series model takes historical corrosion data and environmental parameters (temperature / humidity / medium) as input and outputs a remaining life assessment error of <3.2%; the genetic algorithm optimizes the maintenance strategy with the objective function of minimizing maintenance costs and downtime losses, and the constraint condition is that the risk level is greater than the threshold.
6. A method for intelligent pipeline inspection robot based on multimodal machine learning, applying the system described in claim 5, characterized in that: Includes the following steps: A lightweight detection model is constructed, which achieves efficient feature extraction through HetConv hybrid convolution and ShuffleAttention mechanism, and optimizes small target detection using Inner-SIoU loss function; A multimodal dynamic fusion method is implemented, which fuses visible light and infrared features based on an illumination adaptive weighting strategy, and coordinates with laser data to achieve multi-source information fusion; Establish a federated learning incremental system to achieve dynamic model updates through edge knowledge distillation and cloud-based FedProx aggregation; Design a pipeline adaptive robot, and use a composite support mechanism and DDPG reinforcement learning to achieve contact pressure regulation and path planning; Develop an intelligent decision engine that integrates an LSTM corrosion prediction model with a genetic algorithm maintenance optimization model to complete the tubing condition assessment and maintenance strategy formulation.
7. The method for intelligent pipeline inspection robot based on multimodal machine learning according to claim 6, characterized in that: The specific steps for building a lightweight detection model include: adopting a HetConv hybrid convolutional structure, connecting 3×3 and 1×1 convolutions in parallel to reduce computation by ≥9% and achieve FLOPs≤4.2G; integrating the ShuffleAttention mechanism to enhance the feature extraction capability of oil stain interference through channel grouping and spatial recalibration; designing an Inner-SIoU loss function to focus on gradient updates for small targets, thereby improving the mAP@0.5 index by ≥4.6% in low-light environments, achieving a 2mm corrosion point recognition rate of ≥92.1% and an edge inference latency of ≤80ms.
8. The method for intelligent pipeline inspection robot based on multimodal machine learning according to claim 7, characterized in that: The specific steps for implementing the multimodal dynamic fusion method include: In the feature-level fusion step, the weights of visible light and infrared features are dynamically adjusted using an illumination-adaptive weighted algorithm. The weight value α is dynamically set according to the illumination intensity lux. When lux > 100, α = 0.8; otherwise, α = 0.
3. The fusion formula is as follows: fusion_feat=α*vis_feat+(1-α)*ir_feat; The multi-source data collaboration process employs ResNet18-FPN to extract visible light texture features, UNet to segment infrared temperature anomaly areas: a temperature difference > 5℃ triggers a leak warning, and lidar quantifies the depth of corrosion pits with an accuracy of ±0.1mm.
9. The method for intelligent pipeline inspection robot based on multimodal machine learning according to claim 8, characterized in that: The specific steps for establishing a federated learning incremental system include: an edge-side knowledge distillation step to compress the model to fit the RK3588 NPU; a cloud-based FedProx aggregation step to aggregate gradients from multiple nodes to address the problem of non-independent and identically distributed data; and a two-level update architecture to improve the model's weekly recognition accuracy by ≥0.8%.
10. The method for intelligent pipeline inspection robot based on multimodal machine learning according to claim 9, characterized in that: The specific steps involved in designing a pipeline adaptive robot and developing an intelligent decision engine include: The mechanical structure is innovative, employing a composite support mechanism of helical springs and torsion springs to achieve dynamic adjustment of contact pressure and to overcome obstacles such as 15mm welds and corrosion protrusions. The navigation decision-making steps are based on a path planning model constructed using DDPG reinforcement learning, with the reward function defined as follows: reward = 0.6 information entropy + 0.3 defect density - 0.1 * energy consumption; The corrosion prediction step involves inputting historical corrosion data and environmental parameters (temperature, humidity, and medium) into an LSTM time series model, and outputting a remaining lifetime assessment error of <3.2%. The maintenance optimization steps employ a genetic algorithm with the objective function of minimizing maintenance costs and downtime losses, under the constraint that the risk level is greater than the threshold.