Autonomous cooperative operation and maintenance decision-making method and device for rail transit turnout group

Abstract

The embodiment of the invention provides an autonomous collaborative operation and maintenance decision-making method and device for a rail transit turnout group, and the method comprises the steps: constructing digital twin bodies of the turnout group, and mapping the physical world state of turnout group equipment; respectively modeling into heterogeneous independent agents, storing a global state through a state space, and storing different action sets through an action space; a proposal action is output through the heterogeneous independent intelligent body, the global state and the proposal action are analyzed in real time through a large language model LLM subjected to domain knowledge fine adjustment, the proposal action of the heterogeneous independent intelligent body is evaluated, and a reward signal is generated; a multi-agent near-end strategy optimization MAPPO algorithm is adopted to train each agent strategy network, the trained agent strategy networks are deployed and operated online, a parameterized action space is designed for maintenance actions, and evaluation and reward shaping are carried out on the rationality of operation parameters by utilizing reward signals of LLM, so that the reward performance of the maintenance actions is improved. And finally, outputting an optimal collaborative decision scheme under multiple constraints.

Description

Technical Field

[0001] This document relates to the cross-disciplinary fields of industrial artificial intelligence and rail transit operation and maintenance, and in particular to a method and device for autonomous collaborative operation and maintenance decision-making of rail transit turnout groups. Background Technology

[0002] The turnout system is a vulnerable link in rail transit, and its reliability is crucial. Turnout maintenance and scheduling heavily rely on dispatcher experience, and the current operation and maintenance model faces challenges: 1) Difficulty in resource coordination: There are complex constraints among maintenance tasks, track maintenance windows, manpower, and spare parts, such as overlapping track maintenance windows for multiple lines, and a limited number of highly skilled maintenance workers; 2) Slow dynamic response: Unexpected situations disrupt the original plan, and the plan is difficult to cope with sudden failures, weather changes, and temporary dispatch orders; 3) Single optimization objective: Existing systems focus mainly on shortening time or reducing costs, making it difficult to balance multiple objectives such as efficiency, cost, safety, interference, and long-term effectiveness; 4) Knowledge solidification: It is highly dependent on expert experience, and decision-making knowledge is difficult to accumulate and replicate.

[0003] To solve the above problems, the following technical solution has been adopted in the prior art: 1. Predictive maintenance of single equipment: Fault prediction of a single turnout cannot solve the global optimization problem of multiple turnout maintenance tasks under resource (manpower, track maintenance window time, spare parts, etc.) conflicts, and lacks a group collaboration perspective.

[0004] 2. Static debugging planning: Scheduling systems based on static rules or simple optimization algorithms cannot cope with real-time dynamic changes in the operating environment (such as temporary vehicle additions, sudden weather changes, and sudden equipment degradation).

[0005] 3. Centralized optimization model: suffers from the curse of dimensionality and poor scalability. It attempts to use a central model to schedule all resources, but the model complexity increases exponentially with the number of switches, making it difficult to scale ("curse of dimensionality").

[0006] 4. Low coupling of large models: Large model techniques are mostly applied to question answering and document processing, and are not deeply coupled with decision optimization loops.

[0007] In summary, existing technologies suffer from the following technical problems: they lack the ability to handle real-time dynamics; they cannot perform large-scale distributed optimization while ensuring compliance; and the decision-making process is opaque and difficult to incorporate advanced human intentions.

[0008] Therefore, the following technical problems urgently need to be solved: 1. How to mathematically model complex, multi-objective, and hard-and soft-constraint-laden practical operation and maintenance problems, transforming them into computable MARL problems. 2. How to transform complex rules and safety regulations described in human language into machine-understandable optimization objectives (reward functions), overcoming the fundamental difficulty of manually designing reward functions in MARL. 3. How to achieve fine-grained optimization at the level of maintenance operation parameters, rather than just task scheduling. 4. How to ensure the safety and robustness of the system's decision-making in emergency situations. Summary of the Invention

[0009] The purpose of this invention is to provide a method and apparatus for autonomous collaborative operation and maintenance decision-making of rail transit turnout groups, aiming to solve the above-mentioned problems in the prior art.

[0010] This invention provides a method for autonomous and collaborative operation and maintenance decision-making of rail transit turnout groups, including: Data on turnout group equipment is acquired, a digital twin of the turnout group is constructed, and the physical world state of the turnout group equipment is mapped in real time through the constructed digital twin of the turnout group. The turnout, maintenance unit, and key resources are modeled as heterogeneous independent intelligent agents, and the global state of all heterogeneous independent intelligent agents is stored in the state space, and the different action sets of all heterogeneous independent intelligent agents are stored in the action space. The heterogeneous independent intelligent agent outputs a proposed action, and the global state and the proposed action are analyzed in real time using a large language model (LLM) fine-tuned with domain knowledge. The rationality, safety and / or economy of the proposed action of the heterogeneous independent intelligent agent in the current global state are evaluated, and a reward signal is generated. The MAPPO algorithm for multi-agent proximal policy optimization is used to train the policy networks of each agent. The trained policy networks are then deployed and run online. By designing a parameterized action space for maintenance actions and using the reward signal of LLM to evaluate the rationality of the operation parameters and shape the rewards, the optimal collaborative decision-making scheme under multiple constraints is finally output.

[0011] This invention provides an autonomous collaborative operation and maintenance decision-making device for rail transit turnout groups, comprising: The module is used to acquire data of the turnout group equipment, construct a digital twin of the turnout group, and map the physical world state of the turnout group equipment in real time through the constructed digital twin of the turnout group. The modeling module is used to model turnouts, maintenance units, and key resources as heterogeneous independent intelligent agents, and to save the global state of all heterogeneous independent intelligent agents through the state space and the different action sets of all heterogeneous independent intelligent agents through the action space. The LLM evaluation module is used to output proposed actions through the heterogeneous independent intelligent agent, and to analyze the global state and the proposed actions in real time using a large language model (LLM) fine-tuned with domain knowledge. The module evaluates the rationality, safety and / or economy of the proposed actions of the heterogeneous independent intelligent agent in the current global state and generates a reward signal. The MARL decision module is used to train the policy network of each agent using the MAPPO algorithm for multi-agent proximal policy optimization, deploy and run the trained agent policy network online, design parameterized action space for maintenance actions, and use the reward signal of LLM to evaluate the rationality of operation parameters and shape rewards, and finally output the optimal collaborative decision scheme under multiple constraints.

[0012] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the above-described autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups.

[0013] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the above-described autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups.

[0014] By adopting the embodiments of the present invention, intelligent, autonomous and optimized operation and maintenance of turnout groups can be realized, achieving collaborative optimization under multiple objectives (e.g., time, cost, safety, interference, equipment life, resource utilization and operational reliability). Attached Figure Description

[0015] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention; Figure 2 This is a detailed flowchart of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the principle of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention; Figure 4This is a schematic diagram of the autonomous collaborative operation and maintenance decision-making device for rail transit turnout groups according to an embodiment of the present invention; Figure 5 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0018] Method Implementation Examples According to embodiments of the present invention, an autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups is provided. Figure 1 This is a flowchart of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention, as follows: Figure 1 As shown, the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention specifically includes: Step S101: Obtain data on the turnout group equipment, construct a digital twin of the turnout group, and use the constructed digital twin to map the physical world state of the turnout group equipment in real time; specifically including: Access data sources from SCADA, microcomputer monitoring, vibration sensors, infrared thermal imagers, and MRO (Maintenance, Repair, and Overhaul) management systems to obtain data on turnout equipment. A digital twin of a turnout group is constructed, and the physical world state is mapped in real time through the constructed digital twin of the turnout group. The physical world state includes equipment health, maintenance unit location, resource inventory, track maintenance window time, and train operation schedule.

[0019] The digital twin of the turnout group specifically includes: The system comprises an agent, environment, state, action, and reward. The agent includes a policy network that makes decisions; the environment includes all other turnout agents, maintenance unit agents, resource agents, and the dispatch center; the state is the environmental information perceived by the agent and serves as the basis for its decisions; the action is the executable operation of the agent and is the output of its decisions. The action space is defined as: {Go to A, Go to B, Start maintenance A, Start maintenance B, Request motor, End work, Evacuate}; the reward is the environment's immediate evaluation of the agent's action at a given moment.

[0020] Step S102 involves modeling the turnout, maintenance unit, and key resources as heterogeneous independent intelligent agents, storing the global state of all heterogeneous independent intelligent agents in the state space, and storing the different action sets of all heterogeneous independent intelligent agents in the action space; specifically including: Each turnout, each maintenance unit, and each type of key resource is modeled as a heterogeneous independent intelligent agent, wherein the heterogeneous independent intelligent agents specifically include: turnout intelligent agent, maintenance unit intelligent agent, and key resource intelligent agent; The global state of all heterogeneous independent agents is stored in the state space. This global state includes the health status of each turnout, the location and status of each maintenance unit, the inventory quantity of key resources, track maintenance windows, and weather forecasts. The action space of all heterogeneous independent agents is also stored in the state space. This action space consists of different action sets possessed by each agent. Specifically, the action set of the turnout agent includes: {request maintenance, cancel request, report urgency}; the action set of the maintenance unit agent includes: {go to turnout X, start operation (turnout X, task type), end operation, request resource Y, evacuate}; and the action set of the key resource agent includes: {assigned to maintenance unit U, released}.

[0021] Step S103 involves the heterogeneous independent agent outputting a proposed action, and using a domain-knowledge-adjusted Large Language Model (LLM) to analyze the global state and the proposed action in real time. The rationality, safety, and / or economy of the proposed action under the current global state are evaluated, and a reward signal is generated. Specifically, this includes: The heterogeneous independent intelligence proposes actions based on its own strategy at each decision step; The current global state and the proposed actions of individual heterogeneous independent intelligences are assembled into natural language prompts and input into a large language model (LLM) that has been fine-tuned with domain knowledge. The LLM is deployed on an inference server, and through prompt word engineering and constraint decoding techniques, the output of the LLM is restricted to a predetermined range of safety and rationality criteria. The fine-tuned LLM evaluates the rationality, safety, and economy of the proposed actions of the heterogeneous independent intelligence in the current global state based on its internal knowledge, and outputs a reward signal, wherein the reward signal includes: a comprehensive score and natural language reasoning.

[0022] In this embodiment of the invention, after generating the reward signal, a reward cache is set up to pre-calculate and store the reward signal of LLM. When encountering the same or similar scenario, the reward signal is preferentially read from the reward cache. The reward signal output by LLM is combined with the basic reward to form the final reward signal.

[0023] Step S104 involves training the policy networks of each agent using the MAPPO (Multi-Agent Proximal Policy Optimization) algorithm, deploying and running the trained policy networks online, designing a parameterized action space for maintenance actions, and utilizing the reward signal of LLM (Limited Least Meaning) to evaluate the rationality of the operation parameters and shape the rewards, ultimately outputting the optimal collaborative decision-making scheme under multiple constraints. Specifically, this includes: A multi-agent proximal policy optimization (MAPPO) algorithm is used to train the policy networks of each agent. The MAPPO algorithm employs centralized training and distributed execution. The value function of centralized training is as follows: a centralized critic network is trained, which uses the global state to evaluate the quality of the joint actions of all heterogeneous independent agents, guides the updates of individual actor networks, coordinates agent behavior, and assigns credit. The distributed execution strategy includes: each heterogeneous independent agent has an independent actor network, which makes decisions and updates its policy based on local observations in the global state. The trained policy network is deployed and run online. Heterogeneous independent intelligent agents make decisions in a distributed manner based on real-time observations, and the decisions are aggregated to generate final instructions. The final instructions and the decision reasons generated by the LLM are submitted to the application service layer and reported to the administrator. After obtaining the administrator's review and confirmation instructions, the final instructions are automatically sent to the vehicle terminal or personnel mobile terminal of the maintenance unit through the API interface to obtain maintenance results and subsequent equipment status data. The maintenance results and subsequent equipment status data are fed back to the digital twin of the turnout group for online fine-tuning of the policy network of the digital twin of the turnout group.

[0024] As can be seen from the above steps, the technical solution of this invention constructs a digital twin environment for a turnout group; models turnouts, maintenance units (composite intelligent agents consisting of maintenance teams and fixed resources such as vehicles and tools), and key resources (scarce and mobile resources requiring global scheduling) as heterogeneous and independent intelligent agents; utilizes a domain-knowledge-tuned Large Language Model (LLM) to analyze the system state and agent-proposed actions in real time, generating reward signals that combine safety, economy, and efficiency; employs a Multi-Agent Proximal Policy Optimization Algorithm (MAPPO) based on a centralized training and distributed execution framework to train the policy networks of each agent; by designing a parameterized action space for maintenance actions and using LLM to evaluate the rationality of operating parameters and shape rewards, the optimal collaborative decision-making scheme under multiple constraints is finally output, including maintenance task allocation, personnel scheduling, operating parameter suggestions, and emergency response strategies.

[0025] Multi-agent reinforcement learning (MARL) can theoretically solve the problem of fixed rules in rule-based engines, but its insufficient intelligence and adaptability make it difficult to handle complex, unseen scenarios. Furthermore, its reward function design relies on manual adjustment by experts, making it difficult to characterize complex domain knowledge and limiting its practical application in industrial scenarios. Large Language Models (LLMs), with their rich world knowledge and high-level reasoning capabilities, can precisely compensate for this deficiency of MARL. The domain LLM used in this invention's embodiments derives its knowledge from unstructured text data such as equipment maintenance manuals, historical repair records, safety operating procedures, and expert interview records. Domain knowledge is injected through techniques such as supervised fine-tuning and human feedback reinforcement learning (RLHF). Therefore, the deep integration of LLM and MARL provides a method that can understand human intent, comply with safety rules, and make autonomous optimization decisions, achieving a high degree of methodological integration (LLM+MARL) and a redefinition of the problem.

[0026] The technical solution of this invention reconstructs turnout maintenance from single-point fault prediction into a multi-agent collaborative decision-making problem, simultaneously optimizing multiple objectives such as time, cost, safety, and interference. This invention innovatively utilizes a hybrid architecture of LLM and MARL, leveraging the understanding capabilities of LLM to transform complex and fuzzy domain knowledge into reward signals that MARL can precisely optimize. LLM acts as the "strategic commander," understanding complex constraints and high-level objectives; MARL acts as the "tactical executor," performing distributed real-time decision-making. The decision granularity of this invention extends from maintenance task scheduling to maintenance operation parameter optimization, achieving a balance between short-term benefits and long-term equipment health management. This invention constructs a fully closed-loop autonomous system from state perception to decision execution, where humans are "approvers" rather than "decision-makers," greatly reducing reliance on human experience. This invention directly embeds domain knowledge (rules, safety regulations) as constraints into the MARL reward function design and LLM prompts, ensuring the compliance and safety of decisions.

[0027] The technical solutions of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0028] Figure 2 This is a detailed flowchart of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention, such as... Figure 2The diagram illustrates the macroscopic steps and closed-loop process of the autonomous collaborative operation and maintenance decision-making method of this invention: 1. Data perception and acquisition: The system acquires multi-source data from the physical world. 2. Construction and updating of the digital twin environment: A virtual simulation environment is constructed based on the data, providing a foundation for decision-making. 3. Multi-agent distributed decision-making: Each agent (turnout, maintenance unit, etc.) independently proposes actions based on its own state. 4. LLM reward shaping and evaluation (core innovation): The Large Language Model (LLM) evaluates each proposed action in real time, generates reward signals, and integrates domain knowledge into the optimization process. 5. MARL centralized training and policy optimization: The multi-agent reinforcement learning algorithm uses the rewards provided by LLM to optimize the decision-making strategies of all agents. 6. Scheme generation and human review (safety boundary): The system outputs the optimal decision scheme, which must be reviewed and approved by a human administrator before execution. 7. Instruction issuance and execution: Approved instructions are issued to the field for execution. 8. Result feedback and closed-loop learning: Execution results are fed back to the system for model updates and continuous optimization, forming an autonomous closed loop. Figure 2 The system also incorporates a veto / modification loop for human intervention, ensuring the safety and reliability of the system in practical applications.

[0029] Figure 3 This is a schematic diagram illustrating the principle of the autonomous collaborative operation and maintenance decision-making method for rail transit turnout groups according to an embodiment of the present invention, as shown below. Figure 3 As shown: 1. Data Sensing Layer: Interfaces with various sensors (such as vibration and infrared sensors) and systems (such as SCADA and MRO), responsible for collecting data from the physical world.

[0030] 2. Digital Twin Layer: A virtual mirror of the turnout group built based on the physical model and the data-driven model. It serves as the training and simulation environment for MARL, synchronizing and mapping the physical environment state in real time.

[0031] 3. Agent Decision Layer: Includes an LLM reward model service fine-tuned with domain knowledge (deployed on a high-performance server, with a configurable reward cache library to improve performance), a MAPPO training engine, and deployed agent policy networks (Actors) and a centralized evaluation network (Critic).

[0032] 4. Application Service Layer: Receives decision-making solutions from the decision-making layer, provides a human-computer interaction interface, and visually displays the solutions and the decision rationale generated by the LLM, for human administrators to conduct final review, approval, or rejection. It also provides services such as work order management and API interfaces.

[0033] 5. Execution Control Layer: Interacts with field equipment and personnel terminals to execute final instructions approved by the application service layer.

[0034] A[Data Sensing Layer] SCADA, MRO, Sensors B[Digital Twin Layer] [Virtual Environment and Simulation Platform for Switch Groups] subgraph C [Agent Decision Layer: LLM-MARL Hybrid Architecture] direction TB D[Actor Network] π θ_i a i o i ] E[Agent network] π θ_j a j o j ] F[Centralized Critic Network] V φ s] G[LLM Reward Model Service] H[MAPPO Optimizer] D -- Propose action a_i --> G E -- Propose action a_j --> G B -- Global state S_t --> G G -- Reward signal R_LLM --> H F -- Advantage Â_t --> H H -- Policy gradient ▽θ --> D H -- Policy gradient ▽θ --> E end A -- Real-time data synchronization --> B B -- Global state S_t --> F B -- Local observation o_i --> D B -- Local observation o_j --> E D -- final action a_i -->I E -- Final action a_j --> I I[Application Service Layer] Decision-making solutions and work order suggestions [Includes LLM justification and manual review] J[Execution Control Layer] [Field equipment and personnel terminals] I -- Issue approved instructions --> J J -- Execution result and new state S_t+1 --> B A[Data Sensing and Acquisition] [Multi-source sensor and system data] --> B [Building and updating digital twin environments] Real-time mapping of physical world state] B -->C [Multi-agent distributed decision-making] Each agent proposes actions based on local observations. C -->D[LLM Rewards Shaping and Assessment] [Conduct a safety and rationality review for each action] D -->E[MARL Intensive Training and Strategy Optimization] [Integrating LLM reward-updating agent policy] E -->F {Generate the final decision solution} And submit for manual review} F -- Approval --> G [Instruction Issuance and Execution] [Driving the physical world to perform operational and maintenance actions] G -->H [Result Feedback and Closed-Loop Learning] The execution result will be returned to the digital twin environment. H -->B F -- Reject / Modify --> C Specifically, the data perception layer (A) collects real-time data on device status, resource information, and environmental data from the physical world. This data is synchronized to the digital twin layer (B), constructing a virtual environment that maps in real-time to the physical environment and can be used for simulation and training. The digital twin layer provides the global state (S_t) to the centralized Critic network (F) and distributes local observations (o_i, o_j) to each agent network (D, E). Each agent network proposes action proposals (a_i, a_j) in a distributed manner based on the local observations. These action proposals, along with the current global state (S_t), are concatenated and fed into the LLM reward model service (G). Based on its built-in domain knowledge, the LLM independently evaluates the rationality of each agent's proposed action in the current state, generating a reward signal (R_LLM) and a natural language justification. The MAPPO optimizer (H) receives reward signals from the LLM and advantage estimates (Â_t) from the Critic network, calculates policy gradients (▽θ), and updates the Actor networks of each agent to optimize their decision-making policies. The optimized Actor networks output the final actions, which are then passed to the application service layer (I). This layer transforms the action sequence into specific decision schemes and work order suggestions, along with natural language reasoning generated by the LLM, and provides them to human administrators for final review, approval, or rejection. Approved instructions are then issued to the execution control layer (J), driving on-site physical entities (such as maintenance teams and equipment) to perform operations. The execution results and new state data serve as feedback to the digital twin layer, thus closing the autonomous closed loop of "perception-decision-execution-learning".

[0035] The above processing achieves a deep integration of LLM (responsible for high-level semantic understanding and reward generation) and MARL (responsible for multi-agent distributed decision optimization), and sets up manual review at key decision points to ensure safety and controllability.

[0036] The technical solution of this invention, as an embodiment, specifically includes the following processes: 1. Data Awareness and Twin Construction: 1) Connect to data sources such as SCADA, microcomputer monitoring, vibration sensors, infrared thermal imagers, and maintenance management systems (MRO).

[0037] 2) Construct a digital twin of the turnout group to map the state of the physical world in real time, including equipment health H_i, maintenance unit location L_u, resource inventory R_m, track window time T_window, train timetable, etc.

[0038] 2. Multi-agent modeling and state representation: 1) Agent partitioning: Each turnout (i), each maintenance unit (u) (an independent scheduling unit consisting of a maintenance team and its equipped vehicles, tools and other fixed resources), and each type of critical resource (m) (scarce, mobile resources that require global scheduling, such as specific spare parts) are modeled as heterogeneous independent agents.

[0039] 2) State Space S: The global state is the set of all information: S = {H_i, L_u, R_m, T_window, Weather, ...}. This includes the health status of each turnout, the location and status of each maintenance unit (e.g., busy / idle), the inventory quantity of key resources, track maintenance window time, weather forecast, etc.

[0040] 3) Action Space A: Heterogeneous agents possess different action sets. Each agent's actions include requesting / releasing resources, starting / ending tasks, and reporting status.

[0041] a) Turnout agent A_i: {Request maintenance, cancel request, report urgency} b) Maintenance unit agent A_u: {Go to turnout X, start operation (turnout X, task type), end operation, request resource Y, evacuate} c) Critical resource agent A_m: {Assigned to (maintenance unit U), released} 3. LLM Rewards Shaping: Traditional reward functions require manual design and are difficult to characterize complex intentions. This invention introduces LLM for reward shaping.

[0042] 1) In each decision step, the agent proposes action a based on its own policy π.

[0043] 2) The system assembles the current global state s and the action a proposed by a single agent into a natural language prompt, which is then input into a Large Language Model (LLM) fine-tuned with domain knowledge (such as maintenance manuals and safety procedures). (Note: This is a sequential process; each agent's action is evaluated individually.) 3) The LLM reward model service is deployed on a high-performance inference server, and its output is strictly limited to predetermined safety and rationality criteria through prompt engineering and constraint decoding techniques, effectively avoiding the risk of illusion and ensuring the reliability of the evaluation results. The fine-tuned domain LLM, based on its internal knowledge, evaluates the rationality, safety, and economy of the agent's proposed action in the current global state, outputting a comprehensive score R_LLM (scalar) and a natural language reason.

[0044] 4) As an optional implementation, the system can set up a reward cache library to pre-calculate and store the LLM evaluation results of common (state, action) pairs. When encountering the same or similar scenarios, the reward value is read from the cache first, so as to significantly improve the real-time response performance of the system.

[0045] 5) The output score of the LLM is combined with the basic reward R_basic (e.g., +100 for successful repair) to form the final reward signal, which guides subsequent MARL training: R_total = R_basic + λ * R_LLM (λ is the adjustment coefficient).

[0046] 4. MARL Collaborative Decision Making and Learning: 1) The Multi-Agent Proximal Policy Optimization (MAPPO) algorithm is adopted. MAPPO is an extension of PPO in the multi-agent domain, and its core idea is centralized training (CT) and distributed execution (DEC).

[0047] a) Centralized value function: Train a centralized critic network V_φ(s) that uses the global state s to evaluate the quality of the joint actions of all agents, guide the updates of the individual actor network, coordinate agent behavior, and solve the credit allocation problem.

[0048] b) Distributed strategy: Each agent has an independent actor network π_{θ_i}(a_i|o_i), which makes decisions based on local observations o_i (a subset of the global state s, such as a turnout agent that can only see its own health and the location of nearby maintenance units).

[0049] Policy Update: The update objective of the policy network is to maximize the following pruned alternative objective function to ensure training stability: ```text L^{CLIP}(θ) =L^_t [ min( r_t(θ) Â_t, clip(r_t(θ), 1-ε, 1+ε) Â_t ) ] ``` Where r_t(θ) = π_θ(a_t|s_t) / π_{θ_old}(a_t|s_t) is the probability ratio, Â_t is the Advantage value estimated by the advantage function, and ε is the pruning hyperparameter.

[0050] 5. Decision Implementation and Closed-Loop Feedback: 1) The trained policy network is deployed and run online. Agents make decisions in a distributed manner based on real-time observations, and the system aggregates these decisions to generate final instructions (such as work orders).

[0051] 2) This instruction plan will be submitted to the application service layer and presented to the human administrator in the form of a visual interface or report, along with the decision rationale generated by the LLM. The human administrator has the final right to approve, veto, or modify the plan.

[0052] 3) Only instructions that have been reviewed and confirmed by the administrator can be automatically sent to the vehicle-mounted terminal or personnel mobile terminal of the maintenance unit via the API interface.

[0053] 4) The execution results are fed back to the system, and the maintenance results and subsequent equipment status data are fed back to the digital twin for online fine-tuning of the policy network, forming a closed-loop learning.

[0054] In this embodiment of the invention, the intelligent agent can be decomposed into "five elements". The following is an example of the decomposition of a maintenance unit intelligent agent Agent_U1 (e.g., Wang's team): 1. Main body: Agent_U1 itself, which is a software entity. Its core is a policy network (neural network), which represents the decision-making brain of the maintenance unit.

[0055] 2. Environment: The entire digital twin system for the operation and maintenance of the turnout group, including all other turnout agents, maintenance unit agents, resource agents, dispatch center, etc. The environment is beyond the complete control of Agent_U1.

[0056] 3. Status: The environmental information that Agent_U1 can perceive forms the basis of its decision-making. For example, at time t, its local observation o_t might include: [Switch A health score = 85, Switch B health score = 45, its own position = garage, remaining skylight = 120 minutes, spare parts inventory = 1, ...]. It cannot see the precise location of another maintenance unit U2.

[0057] 4. Actions: The actions that Agent_U1 can perform are the outputs of its decisions. Its action space A is defined as: {Go to A, Go to B, Start repairing A, Start repairing B, Request motor, End work, Evacuate}. At time t, its policy network may output the action Go to B.

[0058] 5. Reward: The environment's immediate evaluation R_{t+1} of Agent_U1's action at time t to go to B. This is the core of learning. For example, after LLM evaluation, it considers "going to the most urgent switch B in the current state is efficient and safe," and therefore gives a reward of +8 points R_LLM. This reward signal will guide Agent_U1's policy network to be more inclined to choose the action of "going to the most urgent switch" in similar situations in the future.

[0059] Detailed breakdown example of LLM+MARL system in turnout group operation and maintenance: This invention provides a specific scenario: A subway line has 3 switches (S_A, S_B, S_C), 1 maintenance unit (U1: Wang's team, including engineering vehicles and tools), and 1 key resource (Motor: switch machine motor, 1 in stock). There is a planned 120-minute maintenance window tonight.

[0060] Step 1: Multi-agent environment modeling (heterogeneous agents) 1. Define intelligent agents: 1) Turnout intelligent agents (3): Agent_S_A, Agent_S_B, Agent_S_C. Core objective: Maintain their own health and avoid malfunctions.

[0061] 2) Maintenance Unit Intelligent Agent (1): Agent_U1. Core Objective: To efficiently complete maintenance tasks and maximize team effectiveness.

[0062] 3) Key Resource Agent (1): Agent_R_Motor. Core objective: Improve its own utilization rate and serve maintenance tasks.

[0063] (Note: Key resource agents are not mandatory, but their introduction can more accurately model resource competition and make the solution better.) 2. Define the state space (S): The global state S_t includes: 1) S_A.health = 85, S_B.health = 45 (precursor signs of failure), S_C.health = 92 2) U1.location = Depot, U1.status = idle 3) Motor.inventory = 1 4) Time.window_remaining = 120 minutes 5) Weather = good 3. Define Action Space A: 1) Agent_S_X: {request_maintenance, cancel_request, do_nothing} 2) Agent_U1: {move_to(S_X), perform_maintenance(S_X), request_part(Motor), complete_task, do_nothing} 3) Agent_R_Motor: {allocate_to(U1), release, do_nothing} 4. Define the reward (Reward R): 1) Basic Reward (R_basic): For example, Agent_U1 gets +100 points for successfully repairing a turnout.

[0064] 2) LLM Reward (R_LLM): Dynamically generated by LLM, used to evaluate and shape agent behavior to conform to complex rules and high-level goals.

[0065] Step Two: The LLM as a "Rewards Coach" Workflow LLM performs a concatenated rationality evaluation of each agent's independent actions based on the global state and provides a reward signal. This avoids the complexity of evaluating joint actions and achieves behavioral coordination among agents through a centralized Critic network and personalized rewards.

[0066] At time t, each agent proposes an action based on its own strategy: Agent_S_B (health score 45) proposes the action request_maintenance.

[0067] Agent_U1 proposes the action move_to(S_B).

[0068] Agent_R_Motor proposes the action do_nothing.

[0069] The system does not evaluate these three actions together, but instead submits each agent's "action + current global state" to the LLM for evaluation sequentially and independently. 1. Evaluate Agent_S_B's action request_maintenance: 1) LLM input (prompt words): "[Role] You are a rail transit operation and maintenance expert. Please evaluate the rationality of the following proposed actions."

[0070] [Status] 110 minutes remaining in the track maintenance window. Turnouts: S_A (85 minutes), S_B (45 minutes), S_C (92 minutes). Maintenance team U1 is in the garage (idle). Motor inventory: 1. Weather: Good.

[0071] [Intelligent Agent] Turnout S_B.

[0072] [Proposed Action] Request maintenance.

[0073] [Evaluation Criteria] Safety, efficiency, necessity. Output a comprehensive score (-10 to 10) and a brief explanation. 2) LLM output (reward shaping): "Rating: +9. Reason: S_B health has dropped to 45 points, far below the safety threshold, posing an extremely high risk of failure. Requesting maintenance immediately is necessary and correct." The system therefore assigns a +9 R_LLM reward to Agent_S_B's request_maintenance action.

[0074] 2. Evaluate Agent_U1's action move_to(S_B): 1) LLM input (prompt words): "[Role] You are a rail transit operation and maintenance expert. Please evaluate the rationality of the following proposed actions."

[0075] [Status] 110 minutes remaining in the track maintenance window. Turnouts: S_A (85 minutes), S_B (45 minutes), S_C (92 minutes). Maintenance team U1 is in the garage (idle). Motor inventory: 1. Weather: Good.

[0076]

Intelligent Agent

[0077] [Proposed Action] Proceed to switch S_B.

[0078] [Evaluation Criteria] Safety, efficiency, and resource utilization. Output a comprehensive score (-10 to 10) and a brief explanation. 2) LLM output (reward shaping): "Rating: +8. Reason: S_B has the highest risk of failure and is the highest priority. Dispatching the only available team, U1, to handle it was an efficient decision. However, it should be noted that S_B may require motor replacement, and preparations should be made in advance." The system therefore assigns a +8 R_LLM reward to Agent_U1's move_to(S_B) action.

[0079] 3. Evaluate Agent_R_Motor's action do_nothing: 1) LLM input (prompt words): "[Role] You are a rail transit operation and maintenance expert. Please evaluate the rationality of the following proposed actions."

[0080] [Status] 110 minutes remaining in the track maintenance window. Turnouts: S_A (85 minutes), S_B (45 minutes), S_C (92 minutes). Maintenance team U1 is in the garage (idle). Motor inventory: 1. Weather: Good.

[0081] [Intelligent Agent] Key Resource (Motor).

[0082] [Proposed Action] No action (maintain current state).

[0083] [Evaluation Criteria] Safety, efficiency, and resource utilization. Output a comprehensive score (-10 to 10) and a brief explanation. 2) LLM output (reward shaping): "Score: 0. Reason: No team has applied to use this resource yet, so keeping it idle is reasonable. However, we should be prepared to respond to any potential motor replacement needs." The system therefore assigns an R_LLM reward of 0 to Agent_R_Motor's do_nothing action.

[0084] Step 3: MARL Collaborative Decision-Making and Learning 1. Each agent makes distributed decisions based on its own policy network (Actor).

[0085] 2. The MAPPO algorithm uses a centralized Critic network (based on global state s) and the total reward R_total provided by LLM to update the policy network of all agents.

[0086] 3. After training, the agents learn cooperative behavior: Agent_S_B will request maintenance in a timely manner; Agent_U1 will tend to go to the most urgent tasks; Agent_R_Motor will prepare to allocate resources when it predicts that Agent_U1 needs them.

[0087] Step 4: Responding to emergencies 1. Status update: Time.window_remaining = 30 min.

[0088] 2. Agent re-decision: Agent_U1's policy network may propose perform_maintenance(S_B).

[0089] 3. LLM Emergency Assessment: 1) Input: ...Sunroof remaining 30 minutes. U1 is already at S_B. Proposed action: U1 [Start maintenance S_B].

[0090] 2) LLM Output: "Score: -7. Reason: Repairing S_B is estimated to take 60 minutes, far exceeding the remaining maintenance window. Forcing the work to begin may result in personnel and equipment being unable to evacuate the line in time, which is a serious violation of safety regulations and carries an extremely high risk. It is recommended to abandon this repair and evacuate immediately." 4. System Response: The negative reward (-7) of LLM forces Agent_U1's policy network to significantly reduce the probability of the perform_maintenance action and choose do_nothing or move_to(Depot). The system ultimately generates a "stop the task and evacuate immediately" instruction and reports the reason through the application service layer.

[0091] This invention transforms the implementation of a single AI model into a well-defined, collaborative "AI team." The LLM (All-Knowing Coach) acts as an expert coach with comprehensive knowledge, while the MARL (Master of Logic) is a highly effective student. This combination enables autonomous, secure, and globally optimal collaborative decision-making in extremely complex real-world environments. This transcends simple "predictive maintenance" and represents autonomous, intelligent operation and maintenance.

[0092] Fine-grained, parameterized action spaces are designed for the maintenance unit's intelligent agent to demonstrate the operational-level optimization capabilities of this invention. For example, for the action of "tightening bolts," the action space is no longer a single perform_maintenance, but is expanded to {tightening bolts (torque = 100 Nm), tightening bolts (torque = 105 Nm), tightening bolts (torque = 110 Nm), ...}.

[0093] The agent's long-term reward G_t is positively correlated with the component's fault-free operating time after the maintenance. During training, when the agent proposes a specific parameter (e.g., 105 Nm), the LLM reward model evaluates the reasonableness of the parameter based on a knowledge base such as the equipment maintenance manual. For example, if the manual recommends a torque of 100-110 Nm, 105 Nm will receive a positive reward, while 150 Nm will receive a negative reward. Through MARL learning, under the precise guidance of the LLM reward signal, the agent will not only learn "what to do" but also "how to do it best," thereby exploring the optimal operating parameters that maximize the long-term reliability of the equipment and achieving refined preventative maintenance.

[0094] In summary, the beneficial effects of the embodiments of the present invention are as follows: 1. Global autonomous collaboration and dynamic balance of multiple objectives: The system can automatically generate the optimal decision scheme that takes into account multiple factors. Thanks to LLM's ability to understand complex, multi-objective and even conflicting domain rules, it can dynamically adjust the optimization weights according to real-time conditions (such as focusing on cost in normal times and speed in emergencies), reducing the reliance on manual scheduling.

[0095] 2. Operational-level optimization and long-term benefits: By designing parameterized actions for the maintenance unit agent (such as perform_maintenance(S_X, Torque=Y)), and having the LLM score the rationality of the operating parameter Y based on the equipment manual knowledge, the system can explore the optimal operating parameters that maximize the long-term reliability of the equipment, achieving deep and refined maintenance from task scheduling to operating parameter optimization.

[0096] 3. Strong anti-interference capability and security: It can quickly replan in the face of emergencies (such as sudden weather changes or temporary compressed window). As a built-in "security auditor", LLM can evaluate the security of decision-making schemes in real time and give huge negative rewards for any action proposal that violates security procedures, thereby fundamentally ensuring the robustness and security of the system's decision-making.

[0097] 4. Knowledge Accumulation, Reuse, and Explainability: LLM transforms expert experience into calculable reward signals, achieving the digitization and automation of knowledge. Simultaneously, the natural language justifications generated by LLM when outputting reward scores (e.g., "Due to the shortened track maintenance window on Line 3, Wang's team will be prioritized to complete the emergency task of the A12 turnout") are stored in the decision log, greatly enhancing the transparency and explainability of the decision-making process and facilitating auditing and traceability.

[0098] 5. Human-Machine Collaboration and Automated Closed-Loop: The system achieves a fully automated closed loop from state awareness to decision suggestion generation. However, the final decision solution generated is submitted to the application service layer for final review and confirmation by a human administrator before execution. This positioning places humans in the role of "approver" and "supervisor," rather than the replaced "decision-maker," ensuring absolute safety boundaries while improving efficiency. The system can directly issue approved work orders to mobile terminals, achieving highly efficient human-machine collaboration.

[0099] Device Example 1 According to an embodiment of the present invention, an autonomous collaborative operation and maintenance decision-making device for rail transit turnout groups is provided. Figure 4 This is a schematic diagram of the autonomous collaborative operation and maintenance decision-making device for rail transit turnout groups according to an embodiment of the present invention, as shown below. Figure 4 As shown, the autonomous collaborative operation and maintenance decision-making device for rail transit turnout groups according to an embodiment of the present invention specifically includes: Construction module 40 is used to acquire data of turnout group equipment, construct a digital twin of the turnout group, and map the physical world state of the turnout group equipment in real time through the constructed digital twin of the turnout group. Modeling module 42 is used to model turnouts, maintenance units, and key resources as heterogeneous independent intelligent agents, and to save the global state of all heterogeneous independent intelligent agents through the state space and the different action sets of all heterogeneous independent intelligent agents through the action space. LLM evaluation module 44 is used to output proposed actions through the heterogeneous independent intelligent agent, and to analyze the global state and the proposed actions in real time using a large language model (LLM) fine-tuned with domain knowledge, to evaluate the rationality, safety and / or economy of the proposed actions of the heterogeneous independent intelligent agent in the current global state, and to generate a reward signal. The MARL decision module 46 is used to train the policy network of each agent using the multi-agent proximal policy optimization MAPPO algorithm, deploy and run the trained agent policy network online, design parameterized action space for maintenance actions, and use the reward signal of LLM to evaluate the rationality of the operation parameters and shape the reward, and finally output the optimal collaborative decision scheme under multiple constraints.

[0100] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operation of each module can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0101] Device Example 2 This invention provides an electronic device, such as... Figure 5 As shown, it includes: a memory 50, a processor 52, and a computer program stored in the memory 50 and executable on the processor 52, wherein the computer program, when executed by the processor 52, performs the steps as described in the method embodiment.

[0102] Device Example 3 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor 52, performs the steps described in the method embodiment.

[0103] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rail transit turnout group autonomous cooperative operation and maintenance decision method, characterized in that, The method comprises the following steps: acquiring turnout group equipment data, constructing a turnout group digital twin, and mapping the physical world state of the turnout group equipment in real time through the constructed turnout group digital twin; modeling the turnout, maintenance unit, and key resources as heterogeneous independent agents respectively, saving the global state of all heterogeneous independent agents through the state space, and saving the different action sets owned by all heterogeneous independent agents through the action space; outputting proposed actions through the heterogeneous independent agents, using a large language model (LLM) fine-tuned by domain knowledge to analyze the global state and the proposed actions in real time, evaluating the rationality, safety, and / or economy of the proposed actions of the heterogeneous independent agents under the current global state, and generating a reward signal; training the agent policy network using a multi-agent proximal policy optimization (MAPPO) algorithm, deploying and running the trained agent policy network online, designing a parameterized action space for maintenance actions, using the reward signal of the LLM to evaluate and reward the rationality of the operation parameters, and finally outputting an optimal collaborative decision-making scheme under multiple constraints.

2. The method of claim 1, wherein, The method of acquiring turnout group equipment data, constructing a turnout group digital twin, and mapping the physical world state of the turnout group equipment in real time through the constructed turnout group digital twin specifically comprises: accessing the data sources of SCADA, microcomputer monitoring, vibration sensors, infrared thermography, and maintenance and operation management system (MRO) to acquire turnout group equipment data; constructing a turnout group digital twin to map the physical world state in real time, wherein the physical world state includes equipment health, maintenance unit location, resource inventory, window time, and train working diagram.

3. The method of claim 1, wherein, The turnout group digital twin specifically comprises: a body, an environment, a state, an action, and a reward, wherein the body includes a policy network for decision-making, the environment includes all other turnout agents, maintenance unit agents, resource agents, and dispatching centers; the state is the environment information that the body can perceive and is the basis for the decision-making of the body; the action is the executable operation of the body and is the output of the decision-making of the body. The action space of the action is defined as: {go to A, go to B, start maintenance A, start maintenance B, apply motor, end work, and evacuate}; the reward is the immediate evaluation of the action of the body at a certain time by the environment.

4. The method of claim 1, wherein, Modeling the turnout, maintenance unit, and key resources as heterogeneous independent agents respectively, and saving the global state of all heterogeneous independent agents through the state space and saving the different action sets owned by all heterogeneous independent agents through the action space specifically comprises: modeling each turnout, each maintenance unit, and each type of key resource as a heterogeneous independent agent, wherein the heterogeneous independent agent specifically comprises: a turnout agent, a maintenance unit agent, and a key resource agent; The global state of all heterogeneous independent agents is saved through the state space, wherein the global state is a set of switch health states, positions and states of each maintenance unit, key resource inventory quantities, weather forecast, and weather forecast; the action space of all heterogeneous independent agents is saved through the state space, wherein the action space is a different action set owned by the heterogeneous independent agents, wherein the action set of the switch agent includes: {request maintenance, cancel request, report emergency degree}, the action set of the maintenance unit agent includes: {go to switch X, start work (switch X, task type), end work, apply for resource Y, withdraw}, and the action set of the key resource agent includes: {be assigned to maintenance unit U, be released}.

5. The method of claim 1, wherein, The proposed action is output by the heterogeneous independent agent, the global state and the proposed action are analyzed in real time by the large language model LLM fine-tuned by domain knowledge, the rationality, safety and / or economy of the proposed action of the heterogeneous independent agent under the current global state are evaluated, and a reward signal is generated, specifically including: The heterogeneous independent agent proposes a proposed action according to its own strategy in each decision-making step; The current global state and the proposed action of the single heterogeneous independent agent are assembled into a natural language prompt, which is input into the large language model LLM fine-tuned by domain knowledge; The LLM is deployed on an inference server, and through prompt word engineering and constraint decoding technology, the LLM output is limited within the predetermined safety and rationality criteria, and the fine-tuned LLM evaluates the rationality, safety and economy of the proposed action of the heterogeneous independent agent under the current global state based on its internal knowledge, and outputs a reward signal, wherein the reward signal includes a comprehensive score and a natural language reason.

6. The method of claim 1, wherein, The method further includes: After generating the reward signal, a reward cache is set up to pre-calculate and store the reward signal of the LLM, and when encountering the same or similar scene, the reward signal is read from the reward cache; The reward signal of the LLM output is combined with the basic reward to form the final reward signal.

7. The method of claim 1, wherein, The multi-agent proximal policy optimization MAPPO algorithm is used to train the agent policy network, the trained agent policy network is deployed and run online, a parameterized action space is designed for maintenance actions, and the rationality of the operation parameters is evaluated and rewarded by using the reward signal of the LLM, and finally an optimal collaborative decision-making scheme under multiple constraints is output, specifically including: The multi-agent proximal policy optimization (MAPPO) algorithm is used to train the policy network of each agent, wherein the MAPPO algorithm is a centralized training and distributed execution, wherein the value function of the centralized training is: a centralized critic network is trained, the critic network uses the global state to evaluate the pros and cons of the joint action of all heterogeneous independent agents, guides the update of the individual actor network, coordinates the behavior of the agents, and performs credit assignment; the policy of the distributed execution includes: each heterogeneous independent agent has an independent actor network, the actor network makes decisions according to the local observation in the global state, and updates the policy. The trained policy network is deployed and run online, the heterogeneous independent agents make decisions in a distributed manner according to real-time observations, and the final instructions are generated after being summarized; the final instructions and the decision reasons generated by the LLM are submitted to the application service layer and reported to the administrator, the administrator's audit confirmation instructions are obtained, the final instructions are automatically issued to the vehicle-mounted terminal or personnel mobile terminal of the maintenance unit through the API interface, the maintenance results and subsequent state data of the equipment are obtained, and the maintenance results and subsequent state data of the equipment are fed back to the turnout group digital twin, and the policy network of the turnout group digital twin is fine-tuned online.

8. An autonomous and cooperative operation decision device for rail transit turnout groups, characterized in that, It comprises: A construction module is configured to obtain turnout group equipment data, construct a turnout group digital twin, and map the physical world state of the turnout group equipment in real time through the constructed turnout group digital twin; A modeling module is configured to model the turnout, the maintenance unit, and the key resources as heterogeneous independent agents, respectively, save the global state of all heterogeneous independent agents in a state space, and save the different action sets owned by all heterogeneous independent agents in an action space; An LLM evaluation module is configured to output proposed actions by the heterogeneous independent agents, analyze the global state and the proposed actions in real time by using a large language model (LLM) fine-tuned by domain knowledge, evaluate the rationality, safety, and / or economy of the proposed actions of the heterogeneous independent agents in the current global state, and generate a reward signal; A MARL decision module is configured to train the policy network of each agent by using the multi-agent proximal policy optimization (MAPPO) algorithm, deploy and run the trained policy network of the agent online, design a parameterized action space for the maintenance action, evaluate and reward the rationality of the operation parameters by using the reward signal of the LLM, and finally output an optimal collaborative decision scheme under multiple constraints.

9. An electronic device, comprising: It comprises: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to implement the steps of the rail transit turnout group autonomous collaborative operation decision method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores an information transmission implementation program, and the program is executed by the processor to implement the steps of the rail transit turnout group autonomous collaborative operation decision method according to any one of claims 1 to 7.

Images