Contact network foreign matter removing system and method based on multi-mode self-adaptive decision
Through the multimodal perception module and hierarchical reinforcement learning decision engine, the environmental robustness and real-time problems of foreign object removal from the contact network in high-speed railways are solved, and the efficient identification and safe removal of foreign objects are achieved to meet the real-time needs of high-speed railways.
Patent Information
- Application Number
- CN202510763050.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-09-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies for clearing foreign objects from contact networks in high-speed railways suffer from insufficient environmental robustness, lack of semantic understanding, poor real-time performance, and waste of resources. They are unable to effectively cope with decision delays and resource waste caused by all-weather scenarios and asynchronous processing of multimodal data.
A multimodal perception module is used to synchronously collect data, and a robust three-dimensional representation of foreign objects is generated through a heterogeneous data fusion network. Combined with a hierarchical reinforcement learning decision engine and a spatiotemporal constraint motion planner, a clearance strategy is dynamically generated to achieve millimeter-level precision robot arm trajectory planning.
It achieves efficient identification and removal of foreign objects under complex meteorological conditions, reduces equipment energy consumption, improves removal efficiency and safety, and meets the real-time needs of high-speed railways. The foreign object detection accuracy rate reaches 98.7%, and the end-to-end delay is ≤180ms.
Smart Images
Figure CN120673335A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intersectional technology between artificial intelligence and rail transit, and in particular to a contact network foreign body removal system and method based on multimodal adaptive decision-making. Background Art
[0002] With the rapid development of high-speed railways, foreign object intrusion into the contact network (such as floating plastic bags and dangling wires) has become a significant threat to train safety. Current mainstream removal technologies suffer from the following limitations: 1. Insufficient environmental robustness: A single sensor cannot cover all weather conditions, and multimodal data lacks deep fusion; 2. Lack of semantic understanding: Traditional algorithms cannot distinguish the properties of foreign objects (conductivity, volume, mass), leading to erroneous decisions; 3. Poor real-time performance: Asynchronous processing of multimodal data results in decision delays (>500ms), making it difficult to meet the requirements of high-speed railways; 4. Waste of resources: Fixed-threshold cleaning strategies (such as full-power laser irradiation) increase energy consumption and equipment wear. Summary of the Invention
[0003] In order to overcome the shortcomings of the existing technology, the present invention proposes a contact network foreign body removal system and method based on multimodal adaptive decision-making. Through the full-link technology of multimodal perception-intelligent decision-making-precise execution, it solves the problems of low removal efficiency and poor safety caused by traditional methods due to data heterogeneity, environmental dynamics and physical constraints.
[0004] The above-mentioned invention objective of this application is achieved through the following technical solutions:
[0005] A contact network foreign body removal system based on multi-modal adaptive decision-making, comprising:
[0006] Multimodal perception module: Synchronously collects contact network environmental data through visible light cameras, infrared thermal imagers, and lidar;
[0007] Heterogeneous Data Fusion Network: This network aligns the feature spaces of visible light, infrared thermal images, and lidar point clouds through an improved cross-modal attention mechanism to generate robust 3D representations of foreign objects.
[0008] Hierarchical reinforcement learning decision engine: Combines a meta-learning pre-trained strategy library with an online Bayesian adaptive mechanism to dynamically generate clearing strategies that adapt to catenary oscillations;
[0009] Spatiotemporal Constrained Motion Planner: This integrates the catenary dynamic deformation model with safety margin constraints to generate millimeter-level precision robotic arm trajectories.
[0010] In a preferred example of the present application: the heterogeneous data fusion network includes a spatial alignment unit, a semantic association unit and a feature fusion unit,
[0011] The spatial alignment unit establishes a mapping relationship between visible light image pixels and lidar point cloud voxels based on the cross-modal attention mechanism, and the calculation formula is: where Q rgb =W q F rgb ,K lidar =W k F lidar ,V lidar =W v F lidar , W q ,W k ,W v is the learnable weight;
[0012] The semantic association unit constrains the semantic consistency of the infrared thermal image and the visible light image through cross entropy loss;
[0013] The feature fusion unit adopts a gating mechanism to dynamically weight multimodal features and outputs a three-dimensional foreign body representation.
[0014] In a preferred example of the present application, the improved ResNet-50 is used to extract local texture features from visible light images in the heterogeneous data fusion network, and the formula is:
[0015] Infrared thermal imaging uses lightweight U-Net to segment the hot spot area and output the heat distribution map: M ir =U-Net(I ir )∈{0,1} H×W ;
[0016] The laser radar point cloud uses PointNet++ to extract geometric features:
[0017] In a preferred example of the present application: the hierarchical reinforcement learning decision engine includes an upper-layer meta-controller, a lower-layer actuator, and an online adaptive unit,
[0018] The upper meta-controller models the environment state transition s based on the graph neural network t+1 =GNN(s t ,a t |θ meta ), where state s t Contains information such as contact network tension, foreign body material, wind speed, etc.
[0019] The lower-level actuator outputs the robot arm motion instructions
[0020] The online adaptive unit uses a Bayesian neural network to evaluate the confidence of the strategy When the Bayesian confidence c t When <0.85, the incremental learning fine-tuning strategy is triggered
[0021] In a preferred example of this application: the hierarchical reinforcement learning decision engine uses the MAML meta-learning framework to pre-train the strategy library, and the optimization goal is where τ i is the i-th training task, and α is the inner loop learning rate.
[0022] In a preferred example of the present application, the spatiotemporal constraint motion planner includes a dynamic deformation modeling unit and a trajectory optimization unit, wherein the dynamic deformation modeling unit solves the finite element equation Kδ=F ext Calculate the contact network deformation δ caused by the manipulator load, where K is the stiffness matrix and F ext is the external force vector;
[0023] The trajectory optimization unit is under safety constraints and stC contact (x t )≤δ max ,||v joint ||≤v safe Generate the robot arm motion trajectory, where x t is the end pose of the robotic arm, v joint is the joint velocity.
[0024] In a preferred example of the present application: the spatiotemporal constraint motion planner verifies the node deformation by improving the RRT* algorithm. If δ(x new )>5mm, the trajectory node is rejected.
[0025] The second object of the present invention is achieved through the following technical solutions:
[0026] A method for removing foreign matter from a contact network based on multimodal adaptive decision-making comprises the following steps:
[0027] S1: Synchronously collect catenary environmental data through multimodal sensors;
[0028] S2: Utilize cross-modal attention mechanism to align visible light, infrared and point cloud feature spaces to generate fused 3D foreign object representation;
[0029] S3: Dynamically generates a clearing strategy based on hierarchical reinforcement learning, triggering online fine-tuning when the decision confidence falls below a threshold;
[0030] S4: Combine the catenary deformation model with safety constraints to plan the motion trajectory of the robotic arm with millimeter-level precision.
[0031] In summary, this application includes at least one of the following beneficial technical effects:
[0032] Through the complementary use of multimodal sensors (visible light / infrared / radar), it overcomes complex weather interference such as rain, fog, and nighttime. Combined with meta-learning pre-training and online Bayesian fine-tuning, it flexibly responds to the properties of foreign objects and the swing of the contact network to achieve dynamic adaptive decision-making. It integrates the contact network deformation model to avoid secondary accidents caused by robotic arm movements. The end-to-end delay is ≤180ms, supports high-speed rail scenarios at a speed of 350km / h, and has a foreign object detection F1-score of 98.7%. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a schematic diagram of a contact network foreign body removal system based on multi-modal adaptive decision-making according to an embodiment of the present application;
[0034] Figure 2 This is a flow chart of a method for removing foreign matter from a contact network based on multimodal adaptive decision-making according to an embodiment of the present application. DETAILED DESCRIPTION
[0035] The present application is further described in detail below with reference to the accompanying drawings.
[0036] In one embodiment, the present application discloses a contact network foreign body removal system based on multi-modal adaptive decision-making, comprising:
[0037] Multimodal perception module: Synchronously collects contact network environmental data through visible light cameras, infrared thermal imagers, and lidar;
[0038] Heterogeneous Data Fusion Network: This network aligns the feature spaces of visible light, infrared thermal images, and lidar point clouds through an improved cross-modal attention mechanism to generate robust 3D representations of foreign objects.
[0039] Hierarchical reinforcement learning decision engine: Combines a meta-learning pre-trained strategy library with an online Bayesian adaptive mechanism to dynamically generate clearing strategies that adapt to catenary oscillations;
[0040] Spatiotemporal Constrained Motion Planner: This integrates the catenary dynamic deformation model with safety margin constraints to generate millimeter-level precision robotic arm trajectories.
[0041] In this embodiment, multimodal sensors (visible light / infrared / radar) complement each other to overcome complex meteorological interference such as rain, fog, and nighttime. Combined with meta-learning pre-training and online Bayesian fine-tuning, it flexibly responds to foreign object properties and contact network swing to achieve dynamic adaptive decision-making, integrates the contact network deformation model, and avoids secondary accidents caused by robotic arm movements. The end-to-end delay is ≤180ms, supporting high-speed rail scenarios at a speed of 350km / h, and the foreign object detection F1-score is 98.7%.
[0042] In one embodiment, the heterogeneous data fusion network includes a spatial alignment unit, a semantic association unit, and a feature fusion unit.
[0043] The spatial alignment unit establishes a mapping relationship between visible light image pixels and lidar point cloud voxels based on the cross-modal attention mechanism, and the calculation formula is: where Q rgb =W q F rgb ,K lidar =W k F lidar ,V lidar =W v F lidar , W q ,W k ,W v is the learnable weight;
[0044] The semantic association unit constrains the semantic consistency of the infrared thermal image and the visible light image through cross entropy loss;
[0045] The feature fusion unit adopts a gating mechanism to dynamically weight multimodal features and outputs a three-dimensional foreign body representation.
[0046] In this embodiment, the problem of spatial misalignment between visible light and point cloud is solved by establishing precise pixel-voxel mapping. The semantic consistency of infrared and visible light is constrained by cross-entropy loss to identify non-heat-generating foreign objects. Dynamic gating weighting improves the robustness of the three-dimensional representation of foreign objects.
[0047] Specifically, the semantic association module (Semantic-CMA) constrains the semantic consistency between infrared images and visible light images through cross entropy loss: Feature-level fusion uses a gating mechanism to dynamically weight multimodal features: F fusion =α·F rgb +β·A spatial +γ·F ir The weights α, β, and γ are generated using a learnable MLP. Verification: Cross-modal retrieval accuracy: A test set of 2,000 samples was constructed and the top-5 retrieval accuracy was calculated (92.1% for this solution vs. 78.3% for the baseline method). 3D positioning error: A laser tracker was used to measure the 3D positioning error of the fused features (mean 2.7 cm, standard deviation 0.8 cm).
[0048] In one embodiment, the visible light image in the heterogeneous data fusion network uses an improved ResNet-50 to extract local texture features, and the formula is:
[0049] Infrared thermal imaging uses lightweight U-Net to segment the hot spot area and output the heat distribution map: M ir=U-Net(I ir )∈{0,1} H×W ;
[0050] The laser radar point cloud uses PointNet++ to extract geometric features:
[0051] In this embodiment, the extraction of local texture features of visible light enhances the ability to identify small foreign objects (such as iron wires), the rapid segmentation of infrared hot spots solves the problem of missed detection of non-heat-generating foreign objects, and the efficient processing of radar point cloud geometric features reduces the computational complexity.
[0052] In one embodiment, the hierarchical reinforcement learning decision engine includes an upper-layer meta-controller, a lower-layer actuator, and an online adaptive unit.
[0053] The upper meta-controller models the environment state transition s based on the graph neural network t+1 =GNN(s t ,a t |θ meta ), where state s t Contains information such as contact network tension, foreign body material, wind speed, etc.
[0054] The lower-level actuator outputs the robot arm motion instructions
[0055] The online adaptive unit uses a Bayesian neural network to evaluate the confidence of the strategy When the Bayesian confidence c t When <0.85, the incremental learning fine-tuning strategy is triggered
[0056] In this embodiment, the upper-level meta-controller models dynamic conditions such as catenary tension and wind speed to predict environmental changes. The lower-level actuator outputs 6-DOF joint angle commands for precise control of the manipulator. Bayesian confidence triggers fine-tuning to address unknown foreign object types, achieving both real-time decision-making and reliability, with a 97.3% success rate in dynamic foreign object removal. Verification: Simulation comparison experiments were conducted in Gazebo, comparing this approach to a traditional Q-learning approach (97.3% success rate vs. 81.2% for the traditional approach). Strategy generation time: Measured on an embedded NVIDIA Jetson AGX Xavier platform, the strategy generation time was ≤800ms.
[0057] In one embodiment, the hierarchical reinforcement learning decision engine uses the MAML meta-learning framework to pre-train the strategy library, and the optimization goal is where τ i is the i-th training task, and α is the inner loop learning rate.
[0058] In this embodiment, rapid scene adaptation is achieved through meta-learning. In the pre-training strategy library of 200+ simulation scenes, the inner loop learning rate α accelerates convergence and optimizes the target formula. It supports incremental learning with few samples, and the adaptation time for new scenarios is shortened to 5 minutes (traditional methods require retraining).
[0059] In one embodiment, the spatiotemporal constrained motion planner includes a dynamic deformation modeling unit and a trajectory optimization unit. The dynamic deformation modeling unit solves the finite element equation Kδ=F ext Calculate the contact network deformation δ caused by the manipulator load, where K is the stiffness matrix and F ext is the external force vector;
[0060] The trajectory optimization unit is under safety constraints and stC contact (x t )≤δ max ,||v joint ||≤v safe Generate the robot arm motion trajectory, where x t is the end pose of the robotic arm, v joint is the joint velocity.
[0061] In this embodiment, finite element equations quantify the load deformation of the manipulator arm, combining constraint deformation with joint velocity to avoid damage to the contact network and ensure the safety and accuracy of the manipulator's movements. Verification plan: Motion accuracy test: The laser tracker measured the end-point repeatability accuracy to ±0.15mm; deformation safety test: The maximum deformation of the contact network under a 10kN tensile force was 3.2mm (safety threshold 5mm).
[0062] In one embodiment, the spatiotemporal constrained motion planner verifies node deformation by improving the RRT* algorithm. If δ(x new )>5mm, the trajectory node is rejected.
[0063] In this embodiment, deformation is verified in real time when nodes are expanded, high-risk trajectories are avoided, the problem of traditional planning algorithms ignoring physical constraints is solved, and the security of motion planning is enhanced.
[0064] In one embodiment, the present application discloses a method for removing foreign matter from a contact network based on multimodal adaptive decision making, comprising the steps of:
[0065] S1: Synchronously collect catenary environmental data through multimodal sensors;
[0066] S2: Utilize cross-modal attention mechanism to align visible light, infrared and point cloud feature spaces to generate fused 3D foreign object representation;
[0067] S3: Dynamically generates a clearing strategy based on hierarchical reinforcement learning, triggering online fine-tuning when the decision confidence falls below a threshold;
[0068] S4: Combine the catenary deformation model with safety constraints to plan the motion trajectory of the robotic arm with millimeter-level precision.
[0069] In this embodiment, cross-modal feature alignment improves the accuracy of foreign object positioning (average three-dimensional error of 2.7 cm), hierarchical reinforcement learning + online fine-tuning reduces decision delay to 180ms (meeting the requirements of 350km / h high-speed rail), and deformation-constrained trajectory planning avoids equipment loss (energy consumption is reduced by 40%).
[0070] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.
Claims
1. A contact network foreign body removal system based on multi-modal adaptive decision-making, characterized in that: include: Multimodal perception module: Synchronously collects contact network environmental data through visible light cameras, infrared thermal imagers, and lidar; Heterogeneous Data Fusion Network: This network aligns the feature spaces of visible light, infrared thermal images, and lidar point clouds through an improved cross-modal attention mechanism to generate robust 3D representations of foreign objects. Hierarchical reinforcement learning decision engine: Combines a meta-learning pre-trained strategy library with an online Bayesian adaptive mechanism to dynamically generate clearing strategies that adapt to catenary oscillations; Spatiotemporal Constrained Motion Planner: This integrates the catenary dynamic deformation model with safety margin constraints to generate millimeter-level precision robotic arm trajectories.
2. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 1, characterized in that: The heterogeneous data fusion network includes a spatial alignment unit, a semantic association unit and a feature fusion unit. The spatial alignment unit establishes a mapping relationship between visible light image pixels and lidar point cloud voxels based on the cross-modal attention mechanism, and the calculation formula is: where Q rgb =W q F rgb ,K lidar =W k F lidar ,V lidar =W v F lidar , W q ,W k ,W v is the learnable weight; The semantic association unit constrains the semantic consistency of the infrared thermal image and the visible light image through cross entropy loss; The feature fusion unit adopts a gating mechanism to dynamically weight multimodal features and outputs a three-dimensional foreign body representation.
3. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 2, characterized in that: In the heterogeneous data fusion network, the improved ResNet-50 is used to extract local texture features from visible light images. The formula is: Infrared thermal imaging uses lightweight U-Net to segment the hot spot area and output the heat distribution map: M ir =U-Net(I ir )∈{0,1} H×W ; The laser radar point cloud uses PointNet++ to extract geometric features:
4. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 1, characterized in that: The hierarchical reinforcement learning decision engine includes an upper-layer meta-controller, a lower-layer actuator, and an online adaptive unit. The upper meta-controller models the environment state transition s based on the graph neural network t+1 =GNN(s t ,a t |θ meta ), where state s t Contains information such as contact network tension, foreign body material, wind speed, etc. The lower-level actuator outputs the robot arm motion instructions The online adaptive unit uses a Bayesian neural network to evaluate the confidence of the strategy When the Bayesian confidence c t When <0.85, the incremental learning fine-tuning strategy is triggered 5. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 4 is characterized by: The hierarchical reinforcement learning decision engine uses the MAML meta-learning framework to pre-train the strategy library, and the optimization goal is where τ i is the i-th training task, and α is the inner loop learning rate.
6. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 1, characterized in that: The spatiotemporal constraint motion planner includes a dynamic deformation modeling unit and a trajectory optimization unit. The dynamic deformation modeling unit solves the finite element equation Kδ=F ext Calculate the contact network deformation δ caused by the manipulator load, where K is the stiffness matrix and F ext is the external force vector; The trajectory optimization unit is under safety constraints and st C contact (x t )≤δ max ,||v joint ||≤v safe Generate the robot arm motion trajectory, where x t is the end pose of the robotic arm, v joint is the joint velocity.
7. The contact network foreign body removal system based on multi-modal adaptive decision-making according to claim 6, characterized in that: The spatiotemporal constraint motion planner verifies node deformation by improving the RRT* algorithm. If δ(x new )>5mm, the trajectory node is rejected.
8. A method for removing foreign matter from a contact network based on multi-modal adaptive decision making, characterized in that: Including steps: S1: Synchronously collect catenary environmental data through multimodal sensors; S2: Utilize cross-modal attention mechanism to align visible light, infrared and point cloud feature spaces to generate fused 3D foreign object representation; S3: Dynamically generates a clearing strategy based on hierarchical reinforcement learning, triggering online fine-tuning when the decision confidence falls below a threshold; S4: Combine the catenary deformation model with safety constraints to plan the motion trajectory of the robotic arm with millimeter-level precision.