AGV full-life-cycle intelligent operation and maintenance method based on twin atlas and reinforcement learning

Abstract

The invention discloses an AGV full-life-cycle intelligent operation and maintenance method based on a twin map and reinforcement learning, and belongs to the technical field of equipment operation and maintenance management, and the method comprises the steps: employing an edge-level dynamic attenuation function and a node-level economic weight operator for aggregation, and obtaining a health vector containing a whole vehicle risk score and a part first-level risk identifier; the method comprises the following steps: training a reinforcement learning agent according to a three-section reward function comprising production line instant income, health improvement income and future risk deduction, and determining an operation and maintenance strategy weight file by adopting a profit-oriented gradient update mode optimization strategy with risk suppression factors; and carrying out equipment operation and maintenance management on the AGV in workshop operation according to the strategy weight file. According to the method, dynamic risk propagation and economic quantitative evaluation are adaptively fused under the real-time working condition of a workshop, and a comprehensive optimal decision of maintenance opportunity and production line income is realized under the condition of non-stop observation through a reinforcement learning strategy closed-loop driving AGV fleet preventive maintenance and scheduling method.

Description

Technical Field

[0001] This invention relates to the field of equipment operation and maintenance management technology, specifically to an intelligent operation and maintenance method for the entire lifecycle of AGVs based on twin graphs and reinforcement learning. Background Technology

[0002] Existing public literature and industry practices generally categorize the maintenance of AGV fleets in discrete manufacturing workshops under the umbrella of digital twins and predictive maintenance. A typical solution first establishes an IoT-based field data acquisition layer, deploying voltage, current, temperature, vibration, and pose sensors on key components such as batteries, motors, drive wheels, and sensors. Raw time-series data is fed back to the backend platform via MQTT, OPC-UA, or industrial Ethernet protocols. The platform typically uses relational or time-series databases for data storage, and then uses rule engines or statistical threshold models to filter out anomalies, generating equipment health alerts. Simultaneously, some systems introduce a 3D visualization digital twin interface, mapping the AGV's structure and operating trajectory to a virtual space, allowing maintenance personnel to view real-time status and historical curves.

[0003] To improve fault prediction accuracy, existing research combines physical modeling with data-driven algorithms to construct mechanism-data hybrid twins. These methods establish simplified models based on kinetic equations, heat conduction equations, or energy balance equations in the offline phase, and then use UKF, extended Kalman filtering, or autoregressive residuals to correct the observed data in the online phase, maintaining consistency between model parameters and field conditions. Some solutions for large-scale fleets also aggregate correlations at the component, system, and vehicle levels into a static topology, using recurrent networks such as LSTM and GRU to perform health scoring on multi-dimensional time series, thereby predicting potential failures in advance.

[0004] In fault propagation and risk assessment scenarios, some literature uses graph databases to store the coupling relationships between AGVs or components and utilizes graph convolutional networks to extract node features. The approach involves defining fixed weights or attenuation coefficients on edges to express the dependencies between electrical, thermal, and control paths as first- or second-order neighborhood operations, achieving joint perception of anomalies across multiple nodes. Traditionally, assessed health indicators are presented using heatmaps, radar charts, or KPI dashboards, supplemented by CMMS-based work order processes, with dispatchers manually deciding when to perform maintenance.

[0005] In the area of ​​automated maintenance decision-making, industrial software has already applied deep reinforcement learning to scheduling optimization. A common implementation method involves constructing a production environment using a discrete event simulator. During the simulation, the action space is discretized into a grid, allowing the agent to select instructions such as maintenance, continued production, or reduced rate. The reward function is typically a linearly weighted combination of output, energy consumption, or maintenance costs. Training algorithms often employ open-source implementations such as DQN, DDPG, or PPO. To reduce model size, the state usually retains only a few features such as component-level average temperature, vibration RMS, or remaining battery power, which are input into the neural network at fixed time steps. The system is then deployed to the field system after the policy converges, and real-time calls are made via an interface.

[0006] While existing solutions can achieve basic health monitoring of AGV equipment through sensor data acquisition, model calibration, and digital twin visualization, their risk identification still mainly relies on static thresholds or fixed-weight graph convolution inference, making it difficult to reflect the cascading impact of real-time component margin decay on downstream nodes. Health assessment results are usually presented as a single KPI, failing to dynamically couple the economic consequences of downtime with the potential for fault propagation, resulting in coarse maintenance priority allocation. At the decision-making level, reinforcement learning training often relies on simplified state vectors and linearly weighted rewards, lacking high-frequency interaction with the health graph after actual deployment. The strategy struggles to converge quickly with changes in production line pace, load fluctuations, or fleet size. When new topologies or business models emerge, parameter retuning or offline retraining is required, resulting in insufficient real-time performance and portability. Furthermore, existing work order processes are mostly manually triggered, failing to establish a closed loop of graph inference, strategy training, and maintenance execution, causing a time lag between risk inference and maintenance actions. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide an intelligent operation and maintenance method for the entire life cycle of AGVs based on twin graphs and reinforcement learning, in order to solve the problem that the existing equipment operation and maintenance methods based on manual triggering have a time lag between risk inference and maintenance actions, which makes the equipment operation and maintenance untimely, affecting the normal use of AGVs and reducing the utilization rate of production lines.

[0008] To achieve the above objectives, the method of this invention integrates digital twins, graph reasoning, and reinforcement learning into a complete closed loop. This allows for the simultaneous assessment of equipment health and the generation of economically optimal maintenance strategies without interrupting production, directly reducing downtime, refining spare parts inventory control, and improving production line utilization. Specifically, the method includes the following steps:

[0009] Step 1: Collect real-time status data of key AGV components during workshop operation and system performance data output by AGV on-board controller, and input the real-time status data and system performance data into a pre-constructed twin map to form a dynamic twin map that runs synchronously with the production line.

[0010] The construction of the twin graph includes: setting key component nodes and system performance nodes, establishing directed edges with influence coefficients and fault triggering thresholds for the power supply, mechanical transmission, heat dissipation and signal coupling relationships between nodes, and writing all nodes and edges into the graph database at once to obtain the twin graph;

[0011] Step 2: Input the dynamic twin graph into the graph neural network and propagate it through the graph neural network once. During the propagation process, record the remaining margin of the upstream node based on the link margin queue, and determine the risk signal triggered by the node overload. Aggregate the risk signal using the edge-level dynamic decay function and the node-level economic weight operator to obtain a health vector containing the whole vehicle risk score and the first-level risk identifier of the component. Normalize and first-order differencing the health vector to construct risk increment data, and combine the number of risky components and the whole vehicle risk score to obtain the economic index. After splicing them into a fixed-length state frame, push it to the reinforcement learning environment.

[0012] Step 3: In the reinforcement learning environment, the digital twin sandbox uses state frames synchronized with the on-site working conditions as the action space, with three types of actions: immediate repair, load reduction, and continued operation, and an optional duration. The reinforcement learning agent is trained based on a three-stage reward function that includes the immediate benefits of the production line, the benefits of health improvement, and the deduction of future risks. The strategy is optimized using a profit-oriented gradient update method with a risk suppression factor until the discounted benefit sequence converges. Then, the operation and maintenance strategy weight file is exported, and the AGV in the workshop is managed according to the strategy weight file.

[0013] Furthermore, in step 1, the key component nodes include battery nodes, motor nodes, lidar nodes, drive wheel nodes, controller nodes, and brake nodes, and the system performance nodes include driving range nodes, positioning accuracy nodes, load margin nodes, and temperature rise margin nodes.

[0014] Furthermore, in step 2, the link margin queue is proposed for three types of critical paths: power supply, mechanical transmission, and heat dissipation. Recording the remaining margin of upstream nodes based on the link margin queue includes: within the same propagation cycle, writing the remaining margin of the upstream node obtained by the difference between the current load of the upstream node and the design margin into the corresponding path queue.

[0015] Furthermore, in step 2, determining the risk signal triggered by node overload includes: within the same propagation cycle, when the difference between the current load of the upstream node and the design margin is greater than 0, it is determined to be a risk signal triggered by node overload, and the record is immediately marked as a risk signal.

[0016] Furthermore, in step 2, the application of a side-level dynamic attenuation function to the risk signal includes: recalculating the dynamic attenuation weight for each directed edge located on the risk signal propagation path, and explicitly introducing the remaining margin of the upstream node and the fault triggering threshold to construct a decreasing function while retaining the static influence coefficient.

[0017] The edge-level dynamic decay function is:

[0018]

[0019] Where, ω ij (τ) represents the directed edge e located on the risk signal propagation path. ij The actual transmission intensity at time τ, γ ij The influence coefficient of curing in step 1; Δr i (τ) represents the upstream node v i The remaining balance at time τ; θ ij For the fault edge e ij The failure trigger threshold for propagation.

[0020] Furthermore, in step 2, the node-level economic weight operator is:

[0021]

[0022] Among them, t k v represents the single repair time of the component corresponding to node k; k The handling output value generated by this component per unit hour; This refers to the node in the dynamic twin graph. In-degree and out-degree in P; k This is the set of all risk propagation edges pointing to this node; e represents the cumulative edge risk intensity currently borne by this node. jk The directed edge for risk propagation points to node k.

[0023] Furthermore, in step 2, the economic index obtained by mapping the number of risky items and the overall vehicle risk score includes:

[0024] A mapping table is used to convert the set of primary risk identifiers into downtime cost estimates, and then the vehicle risk score is converted into production line efficiency loss rate. The economic index is determined based on the downtime cost estimates and the production line efficiency loss rate.

[0025] Set of Level 1 Risk Labels Converted into estimated downtime cost C pre The piecewise function is:

[0026]

[0027] Among them, c d,jLet ζ represent the cost of a single downtime for component j, and ζ be the threshold for increasing the number of risky components.

[0028] Based on the estimated downtime cost C pre and production line efficiency loss rate D los Determine the economic index E idx The piecewise function is:

[0029]

[0030] Where α and β are the weights of maintenance costs and production line revenue, γ is the risk exceeding threshold penalty coefficient, and R thr R is the risk score penalty threshold. veh This is used to score the overall vehicle risk.

[0031] Furthermore, in step 3, the three-stage reward function, which includes immediate production line revenue, health improvement revenue, and future risk deduction, involves designing the reward signal as a three-stage structure of immediate production line revenue, health improvement revenue, and cascading risk deduction.

[0032]

[0033] Among them, v u Q represents the output value per unit of transportation. mat v represents the effective handling volume for the current period. u Q mat This represents the immediate revenue of the production line, where ΔR is the difference between the previous period's score and the current overall vehicle risk score, indicating the degree of health improvement. l E is the downtime cost factor used to convert the improvement in health into economic benefits, δ is the discount rate for future losses, and E is the downtime cost factor. idx γ is the excess risk penalty coefficient used to estimate future downtime losses.

[0034] Furthermore, in step 3, the profit-oriented gradient update method with risk mitigation factors includes:

[0035] Write the state sequence, action sequence and reward sequence collected by the agent in a batch into the playback buffer;

[0036] For each sequence, first calculate the discounted cumulative revenue, then read the estimated downtime loss sequence of the same batch, and the difference between the two is used to obtain the net profit trajectory;

[0037] The average of all net profit trajectories is used as the economic baseline. Then, the economic advantage at each step is calculated by subtracting the net profit trajectory at each step from the economic baseline, and multiplied with the logarithmic probability gradient of the corresponding action to generate the profit-oriented gradient.

[0038] At the same time, the frequency of occurrence of primary risk nodes is counted, and the frequency of occurrence of primary risk nodes is converted into a suppression factor by subtracting the frequency of occurrence of primary risk nodes from 1. The gradient is scaled by the suppression factor to form a set of risk suppression gradients.

[0039] The update vector is obtained by summing each gradient of the risk suppression set. It is then applied to the policy network parameters with a fixed step size. After the update is completed, the high-load scenario is immediately replayed in the digital twin sandbox for verification: if the average net profit is not lower than the previous update and the cumulative duration of the first-level risk has not increased, the parameters are confirmed to be effective; otherwise, the old weights are rolled back and the fixed step size is reduced before iterating again.

[0040] Furthermore, in step 3, the equipment operation and maintenance management of AGVs in the workshop according to the strategy weight file includes: packaging the strategy weight file and the action mapping table into a lightweight operation package and uploading it to the workshop private cloud decision node; after receiving the real-time status frame, calling the strategy function to generate preventive maintenance instructions containing action codes, planned downtime and spare parts requirement list, and sending them in parallel to the AGV controller and operation and maintenance management system to complete automatic execution and work order closure.

[0041] The method of the present invention has the following advantages:

[0042] Real-time monitoring data is incrementally written and then propagated along the edges in real time, which instantiates the static structure into a dynamic twin graph that evolves synchronously with the production line clock. This allows topological constraints and state evolution information to be retained simultaneously within the same graph domain, which is different from the traditional approach of managing two types of information only with timing tables or independent topology files.

[0043] In terms of assessment, this invention introduces a link margin queue and a dynamic edge-level attenuation function. It utilizes the remaining margin of upstream nodes to attenuate edge risk signals in segments. Simultaneously, a product-type economic weight operator is constructed at the node level to couple downtime, unit output value, node degree, and cumulative edge-level risk intensity in real time. A single propagation through the graph neural network outputs the overall vehicle risk score and the set of first-level risk nodes, achieving simultaneous updates of risk quantification, economic quantification, and structural quantification. This overcomes the limitations of fixed-weight or linear threshold models in capturing chain effects.

[0044] In decision-making and execution, a three-stage reward function is adopted to drive the reinforcement learning agent, which includes immediate production line revenue, health improvement revenue, and future risk deduction. In the gradient update, a dual weighting mechanism is introduced, with net profit as the baseline and the frequency of occurrence of first-level risks as the suppression factor, to ensure that strategy optimization takes into account both economic goals and risk suppression.

[0045] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall process of the intelligent operation and maintenance method for the entire life cycle of AGV in this embodiment;

[0047] Figure 2 This is a schematic diagram of the reasoning and risk diffusion assessment process in the method of this embodiment;

[0048] Figure 3 This is a schematic diagram comparing the method of this embodiment with the prior art. Detailed Implementation

[0049] The technical solution of the present invention will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art should understand that the embodiments described below are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] An Example of an Intelligent Operation and Maintenance Method for the AGV Full Lifecycle Based on Twin Graphs and Reinforcement Learning

[0051] This invention aims to provide a preventive maintenance and scheduling method for AGV fleets that can adaptively integrate dynamic risk propagation and economic quantitative assessment under real-time workshop conditions and be driven by a closed-loop reinforcement learning strategy, thereby achieving a comprehensive optimal decision on maintenance timing and production line benefits without stopping the machine for observation.

[0052] like Figure 1 As shown, this embodiment includes the following steps:

[0053] S1. Decompose a single AGV in operation in the workshop into key component nodes such as battery node, motor node, lidar node, drive wheel node, controller node and brake node, as well as system performance nodes such as driving range node, positioning accuracy node, load margin node and temperature rise margin node. Establish directed edges with influence coefficients and fault triggering thresholds for the power supply, mechanical transmission, heat dissipation and signal coupling relationship between nodes, and write all nodes and edges into the graph database at one time to generate a structured heterogeneous twin graph.

[0054] S2. Real-time acquisition of voltage, current, speed, temperature, position and load rate measurements of each component, as well as real-time performance values ​​of driving range, positioning accuracy, load margin and temperature rise margin output by the vehicle controller. Incremental data is written into the twin graph according to the unique node identifier. Real-time propagation is performed along the preset directed edge for nodes with attribute changes to obtain a dynamic twin graph that runs synchronously with the production line.

[0055] S3. Input the dynamic twin graph into the graph neural network, and complete one information propagation according to the order of power supply, transmission, heat dissipation and signal. During the propagation process, record the remaining margin of the upstream node based on the link margin queue. Aggregate the risk signal with the edge-level dynamic attenuation function and the node-level economic weight operator to generate a health vector containing the whole vehicle risk score and the component first-level risk identifier.

[0056] S4. Normalize and first-order differ the health vector to construct risk increment data, and combine the number of risky items and the overall vehicle risk score to obtain an economic index. After splicing the data into a fixed-length state frame, push it to the reinforcement learning environment.

[0057] S5. In the digital twin sandbox, for the state frames synchronized with the on-site working conditions, the action space is based on three types of actions: immediate repair, load reduction operation, and continued operation, as well as the optional duration. The reinforcement learning agent is trained according to a three-stage reward function that includes the immediate benefits of the production line, the benefits of health improvement, and the deduction of future risks. The strategy is optimized by a gradient update method with profit orientation and risk suppression factors until the discounted benefit sequence converges and the strategy weight file is exported.

[0058] S6. Package the strategy weight file and action mapping table into a lightweight runtime package and upload it to the workshop private cloud decision node. After receiving the real-time status frame, call the strategy function to generate preventive maintenance instructions containing action codes, planned downtime and spare parts requirement list, and send them to the AGV controller and operation and maintenance management system in parallel to complete automatic execution and work order closure.

[0059] Step S1 includes the following sub-steps:

[0060] S11. Disassemble a single AGV in operation in the workshop into battery nodes v bat Motor node v mot LiDAR node v lid Drive wheel node v drv Controller node v ctl Brake node v brk Six categories of key physical component nodes; for each node, write a unique identifier (UID), a factory serial number (SN), a design rating (r), and a real-time status placeholder field (s(t)), and consolidate them into a unified template (T). part .

[0061] S12, The vehicle-level range node p rng Positioning accuracy node p loc Load margin node p ld Temperature rise margin node p thm The four system performance indicators are abstracted into independent performance nodes; each performance node is bound to a measurement method m and an operating threshold range.<L,U> And the economic weight λ, and label the calculation result field in the node as z(t).

[0062] S13. Enumerate the physical coupling relationships between each node one by one, and define the supply and demand actions such as battery to motor, controller to drive wheel, controller to brake, etc. as directed edges of the energy transfer or mechanical drive class e. + .

[0063] Secondly, the reverse effects of drive wheel wear feedback to motor load and motor temperature rise feedback to battery discharge rate are defined as directed edges of thermal coupling or signal acquisition type e. - .

[0064] Each edge is assigned an influence coefficient γ and a fault triggering threshold θ to characterize the strength of fault propagation along the edge and the initiation conditions.

[0065] S14. Using a graph database oriented towards attribute graphs, the set of all nodes V = {v bat ,v mot ,v lid ,v drv ,v ctl ,v brk ,p rng ,p loc ,p ld ,p thm} and the set of all edges Write in one go to generate a structured heterogeneous twin map G0.

[0066] Step S2 includes the following sub-steps:

[0067] S21. The original measured values ​​of voltage, current, speed, temperature, pose and load rate of the five monitoring points installed at the battery, motor, drive wheel, lidar and controller are summarized into a single vector η(τ), where τ represents the timestamp; according to the node's unique identifier UID, each component in η(τ) is directly written into the real-time status field s(τ) of the corresponding component node, and τ is written at the same time to lock the record order.

[0068] S22. Encapsulate the four real-time performance values—vehicle-level driving range, positioning accuracy, load margin, and temperature rise margin—built into the vehicle controller into a polling vector π(τ); perform item-by-item mapping on π(τ) and write them into the performance node p. rng p loc pld p thm The instantaneous result field z(τ); based on the one-way reference relationship established in S12, the latest s(τ) of all component nodes is retrieved.

[0069] S23. Generate an update set U(τ) for all nodes whose attributes have changed since the last commit; write U(τ) into the graph database as a single batch transaction to ensure that only the nodes that have changed are touched; push the list of nodes contained in U(τ) and their latest attributes to the upper-level inference.

[0070] S24. Perform an instantaneous propagation of the updated node values ​​along the preset directed edges in the graph. For each edge, read its influence coefficient γ and perform instantaneous effect calculation to obtain a unified state graph G(τ) covering all nodes in V; mark G(τ) with short-period retention identifiers. Based on this, a static twin graph is obtained. Instantiated into a dynamic twin graph via incremental propagation The unified state view generated by a single instantaneous propagation is denoted as H(t), and its short-period retention window is τ. ret Labeling. Subsequent reasoning and strategy learning are all based on... For input.

[0071] like Figure 2 As shown, step S3 includes the following sub-steps:

[0072] S31. The latest dynamic twin graph output in step S2. Import the whole thing; The set of all nodes V within the node array is checked for omissions; the set of opposite edges is checked. Compare the source and destination identifiers one by one to verify the direction and the predefined physical path in S13; randomly check whether the static influence coefficient γ of each side is within the modeling allowable range <0,1>.

[0073] S32, will Direct Input Graph Neural Network (GNN) phy Information is propagated once for all physical connections in the order of power supply, transmission, heat dissipation, and signal. During the propagation process, state transition is completed only by relying on the predetermined direction and influence coefficient γ on the edge, ensuring that the inference link corresponds to the electrical path, mechanical path, heat dissipation path and control path inside the AGV, and obtaining the latest load level of each node of the whole vehicle within a single wheel propagation.

[0074] S33. For the three critical paths of power supply, transmission, and heat dissipation, a link margin queue Q mechanism is proposed; within the same propagation cycle, the current load of the upstream node is... up With design margin r up The difference Δl=l up -r upWrite to the corresponding path queue; if Δl>0, immediately mark the record as a risk signal, and proceed accordingly. The dependency relationship is synchronously pushed into the downstream node queue to explicitly capture the chain trend of motor temperature rise after the drive wheel wear reaches its limit, which in turn induces the battery discharge pressure to increase.

[0075] S34. For risk signals triggered by node overload, this application proposes to use an edge-level decreasing and node-level strategy to aggregate and construct weight calculations.

[0076] For each directed edge e located on the risk signal propagation path ij Recalculate the dynamic decay weight ω ij (τ), while retaining the static influence coefficient γ ij Based on this, the remaining margin Δr of the upstream node is explicitly introduced. i (τ) and the fault trigger threshold θ ij Construct a decreasing function:

[0077]

[0078] Where, ω ij (τ) represents edge e ij The actual transmission intensity at time τ; γ ij Δr is the static influence coefficient for S13 curing. i (τ) represents the upstream node v i The remaining balance at time τ; θ ij For the fault edge e ij The trigger threshold for propagation.

[0079] The piecewise function ensures that the risk signal on the edge only occurs in Δr. i (τ) continues to decay and propagate downstream when it is below the threshold. Once the remaining amount is restored to the safe range, it is actively truncated to achieve dynamic self-inhibition at the edge level.

[0080] At the node level, calculate the economic weight w for node k that has been hit by the risk signal. k Taking into account both the direct costs of downtime and the potential for cascading effects, an improved product operator is proposed:

[0081]

[0082] Among them, t k v represents the single repair time of the component corresponding to node k; k The handling output value generated by this component per unit hour; For each node k, in In-degree and out-degree in P; k This is the set of all risk propagation edges pointing to this node; Let e ​​be the cumulative edge risk intensity currently borne by node k. jk The directed edge for risk propagation points to node k.

[0083] First, the downtime is multiplied by the output value to obtain the direct economic loss. Then, it is multiplied by the node degree to measure the potential diffusion range. Finally, the total amount of real-time risk after edge reduction is superimposed to achieve three-dimensional coupling of cost, structure and real-time risk (i.e., achieving cost × structure × real-time risk).

[0084] According to w k All nodes are sorted in descending order, and divided into Level 1, Level 2 and Security Zones by 1 / 3. The level labels are written back to the corresponding nodes and synchronously accumulated along the performance convergence edge to the four system performance nodes, thus injecting the economic consequences of downtime and the chain reaction trend into the system-level risk profile.

[0085] S35, For the four performance nodes p rng p loc p thm p ld Perform an upward aggregation, multiplying the cumulative risk amount of each component by the production line downtime cost coefficient λ. line The weighted summation yields the single vehicle risk score R. veh ;

[0086] Simultaneously read the c of all first-level risk nodes in the link margin queue. d Calculate the sum of these values ​​as the downtime cost prediction C. pre It is used to reflect the potential economic losses if the current risks are not intervened, so that the health assessment results can directly correspond to subsequent operation and maintenance investment decisions.

[0087] S36. Calculate the overall vehicle risk score R veh Compared with the original values ​​of the four performance nodes [z] rng ,z loc ,z thm ,z ld ] Concatenate into a fixed-length health vector h = [R veh ;z rng ;z loc ;z thm ;z ld Add a unique vehicle identifier (UID) to the tail of the vector. veh Write the timestamp τ to the buffer.

[0088] Step S4 includes the following sub-steps:

[0089] S41. The health vector h = [R] output in step S3. veh ;z rng ;z loc ;z thm ;z ld ;UIDveh ;τ] is split into whole vehicle risk score R according to the meaning of the field. veh Four performance original values ​​{z rng ,z loc ,z thm ,z ld} and component level-one risk identification set Three sets of data; for R veh Each of the four performance parameters has an upper and lower bound set according to industry experience. <R min ,R max >and Normalization was performed using the linear scaling method, resulting in... and Reassembled into a 6×1 fixed-length state matrix using a fixed column order.

[0090] S42, regarding the state matrix S of the previous period (-1) With the current period S (0) Perform a first-order difference to generate the risk increment matrix ΔS = S (0) -S (-1) ; Concatenate ΔS and S row by row (0) This constitutes a 12×1 two-layer state frame F = [S (0) ;ΔS];

[0091] The structure retains both the static risk level and the dynamic rise and fall trend, enabling the reinforcement learning agent to capture the direction of change as soon as the risk shows signs of rising, provided that the Markov assumption is satisfied.

[0092] S43. Furthermore, a mapping table is used to set the primary risk identifiers. Converted into estimated downtime cost C pre Then, the overall vehicle risk score R is calculated. veh Converted to production line efficiency loss rate D los ;

[0093] For these two types of economic quantities, this step implements the estimation, discounting, and composite mapping using a piecewise function:

[0094]

[0095] Where: c d,j E represents the cost of a single downtime for component j; ζ is the risk factor threshold used to distinguish between localized failures and large-scale spread. idx The final economic index; α and β are the weights of maintenance costs and production line revenue, respectively; γ is the risk exceeding threshold penalty coefficient; R thr This is the threshold for risk score penalty.

[0096] Based on the above formula, a continuous mapping is achieved between fault scale, economic loss, and risk penalty, and the result E is plotted.idx As a single economic index field added to the tail of F, the agent can simultaneously weigh maintenance costs and production benefits during policy learning, ensuring that the decision-making logic aligns with the workshop's economic objectives.

[0097] S44. Push the state frame F after field expansion to the reinforcement learning environment; return the unique index number σ(τ) in the message header for graph inference.

[0098] Step S5 includes the following sub-steps:

[0099] S51. Input the status frame F pushed in step S4 into the digital twin sandbox S. tw ; in S tw Internally, the operating condition curve L is consistent with that of the actual workshop. op (t) and parameter ρ cyc Apply a synchronization load to the virtual AGV so that the status of each frame is fed back in the physical environment; at the same time, set the sandbox clock t sim Lock to the field clock t phy This ensures that the virtual and physical timelines remain aligned in real time during strategy training.

[0100] S52. The dimensionality of maintenance behaviors executable in the sandbox is reduced, limiting the action space to three types of atomic instructions: Immediate Repair (ACT). fix ACT (Active Controlled Load) derate Continue running ACT keep ; Add an optional duration parameter d to each type of instruction. act ∈{5,10,15} (minutes); Implement instruction mapping table Φ on the sandbox side. act Action encoding (ID) from the agent act ,d act It can parse and map to the corresponding load scheduling and virtual maintenance processes.

[0101] S53. Furthermore, this application constructs an improved reward function, which designs the reward signal as the immediate revenue of the production line. Health Improvement Benefits Chain risk deduction Three-section structure.

[0102] Effective transport volume Q for sandbox return mat Multiply by unit output value v u To obtain immediate benefits from the production line;

[0103] Read the current vehicle risk score Compared to the previous period's score The difference If ΔR > 0, then the health improvement is considered valid; finally, the economic index E at the tail of the state frame is directly taken. idxAs an estimate of future downtime losses:

[0104]

[0105] Where: v u The value of transported goods per unit; Q mat c is the effective handling volume for the current cycle. l δ is the downtime cost factor, used to convert the improvement in health into economic benefits; E is the discount rate for future losses; idx It is used to estimate future downtime losses; γ is the excess risk penalty coefficient.

[0106] The process involves first calculating immediate output value, then adding or deducting rewards based on changes in health status, and finally uniformly discounting future risks to complete the synthesis. This avoids imbalances in reward signals due to inconsistent dimensions during training. Simultaneously, the three conditions correspond to business scenarios of decreasing risk, unchanged risk, and increasing risk, enabling the agent to dynamically balance production line revenue and chain safety, ensuring that the learning objectives always comply with financial constraints.

[0107] S54. At the beginning of each training round, write the latest state frame F into the reinforcement learning environment. After the agent reads F, it applies the current policy π. θ Selected Action (ID) act ,d act Sandbox S tw Instantaneous access to the mapping table Φ act Perform the corresponding maintenance or scheduling, and refresh the status F in the next frame. next Simultaneously, return the reward R calculated according to rule S53. step .

[0108] S55. Furthermore, this application proposes an improved gradient update method:

[0109] The state sequence {F} collected by the agent in a batch i}、Action sequence{(ID) act ) i} and reward sequence {R step,i Write to playback buffer

[0110] For each sequence, first calculate the discounted cumulative return G. i Then read the estimated downtime loss sequence {C} from the same batch. pre,i The difference between the two yields the net profit trajectory {P}. i};

[0111] For all {P i Calculate the mean as the economic baseline. Then calculate the economic advantage at each step. And multiply it with the corresponding action log probability gradient to generate a profit-oriented gradient;

[0112] Simultaneously, the frequency f of occurrence of primary risk nodes is statistically analyzed. risk It is converted into an inhibitory factor (1-f) risk The gradients are scaled to form a risk-suppressed gradient set;

[0113] The update vector g is obtained by summing each element in the set, and then applied to the policy network parameters θ with a fixed step size η; after the update is completed, it is immediately applied to S. tw Mid-playback high-load scenario verification: If average net profit Not less than the last update and the cumulative duration of Level 1 risk If the value is not increased, the parameter is confirmed to be effective; otherwise, it is rolled back to the old weight and η is reduced to iterate again, achieving convergence with profit improvement and risk suppression.

[0114] S56. When the cumulative discounted return sequence is in continuous N... conv Fluctuations within the round of evaluation do not exceed the threshold ∈ conv That is, determine if the strategy has converged; set the network weights θ at this point. * Export as a lightweight file WGT opt And package it to generate an operation and maintenance deployment package PKG. opt ;

[0115] Step S6 includes the following sub-steps:

[0116] S61. Convert the optimal operation and maintenance strategy weight file WGT output in step S5. opt With action mapping table Φ act Packaged as a lightweight runtime package PKG opt ; Regarding PKG opt Calculate the hash value h pkg Generate version number v pkg And set the effective time window. <t start ,t end > Uploaded to the workshop private cloud decision node NODE dec Write the following to the successful upload receipt: <h pkg ,v pkg This ensures that subsequent instructions are traceable and rollbackable; at the same time, a policy mapping buffer B is reserved inside each AGV onboard controller. map Persistent storage of runtime package index idx pkg and the URL of the API call pkg .

[0117] S62. Send the health status frame F generated in real time in step S4 through the message queue MQ. hlth Enter NODE dec ; Decision node calls PKG optThe internal strategy function πθ* is based on the vehicle risk score R in F. veh Economic Index E idx With component level one risk identification set Calculate recommended actions (ID) act ,d act Assembly action coding act Planned downtime t plan Spare parts requirement list (SP) need For a single preventative maintenance command (CMD) pm And via the workshop dispatch bus sch Push notification.

[0118] S63, CMD pm Parallel transmission to AGV controller CTRL veh With Operation and Maintenance Management System (OMS); CTRL veh Automatically perform speed reduction, offline, or standby operations within the earliest safe window, and record the actual start and end times of shutdown (t). real OMS is based on SP need Generate work order (WO) id Match available technicians with spare parts inventory, and integrate with the production line in the intelligent scheduling terminal TERM sch The above specifies the work period; after the maintenance is completed, press CTRL. veh Returning actual downtime Δt real OMS reports actual spare parts consumption SP used Both are written into the graph database, and the node history is refreshed synchronously.

[0119] This application first writes physical component nodes such as batteries, motors, and drive wheels, along with system performance nodes such as driving range and positioning accuracy, into an attribute graph database at the data and model layer. It then explicitly encodes various coupling relationships, including power supply, transmission, heat dissipation, and signals, as directed edges with static influence coefficients and fault trigger thresholds. Real-time monitoring data is incrementally written and then propagated along the edges in real-time, instantiating the static structure into a dynamic twin graph that evolves synchronously with the production line clock. This allows for the simultaneous preservation of topological constraints and state evolution information within the same graph domain, a feature distinct from traditional methods that manage both types of information using only time-series tables or independent topology files.

[0120] In terms of assessment, this scheme introduces a link margin queue and a dynamic edge-level attenuation function. It utilizes the remaining margin of upstream nodes to attenuate edge risk signals in segments. Simultaneously, a product-type economic weight operator is constructed at the node level to couple downtime, unit output value, node degree, and cumulative edge-level risk intensity in real time. A single propagation through the graph neural network outputs the overall vehicle risk score and the set of first-level risk nodes, achieving simultaneous updates of risk quantification, economic quantification, and structural quantification. This overcomes the limitations of fixed-weight or linear threshold models in capturing chain effects.

[0121] In decision-making and execution, a three-stage reward function is adopted to drive the reinforcement learning agent, which includes immediate production line revenue, health improvement revenue, and future risk deduction. In the gradient update, a dual weighting mechanism is introduced, with net profit as the baseline and the frequency of occurrence of first-level risks as the suppression factor, to ensure that strategy optimization takes into account both economic goals and risk suppression.

[0122] like Figure 3 As shown, compared to traditional predictive operation and maintenance schemes that overlay static topology files with independent time-series databases, this application stores node attributes and coupling relationships uniformly in a heterogeneous twin graph, achieving real-time consistency between topology constraints and state evolution. The link margin queue, combined with the edge-level decreasing function, can quantify cascading risks in a single round of information propagation, thereby accurately characterizing the cascading effects between components. On the other hand, the product-type economic weight combines downtime, unit output, and node degree, introducing an economic dimension for evaluation. The three-stage reward and risk suppression gradient enable the reinforcement learning strategy to simultaneously consider both returns and safety, exhibiting stronger adaptability and transferability than linear weighted models.

[0123] This invention integrates digital twins, graph reasoning, and reinforcement learning into a complete closed loop, enabling simultaneous assessment of equipment health and generation of economically optimal maintenance strategies without interrupting production. This directly reduces downtime, refines spare parts inventory control, and improves production line utilization, resulting in significant cost savings and efficiency improvements. The method has migration potential for any system with a defined physical topology, real-time status monitoring, and high maintenance costs, such as semiconductor cleanroom transportation, automated warehouses, unmanned trucks in ports, elevator group control, and wind farm turbine maintenance. It can be quickly deployed by changing node types and coupling relationships, enabling operators to achieve precise preventative maintenance capabilities based on an economic perspective in large-scale equipment networks.

[0124] To demonstrate the effectiveness of the AGV full lifecycle intelligent operation and maintenance method in this embodiment, the following experimental procedure was conducted:

[0125] After expanding its production line, an assembly plant introduced thirty 1.2t-class holstering AGVs to transport material frames between the SMT line and the aging line. Before adopting the method described in this embodiment, due to the drastic load fluctuations under the day and night double-shift system, battery over-discharge, motor overheating, and abnormal wear of the drive wheels occurred frequently, resulting in an average annual downtime of over 240 hours.

[0126] After using the method of this embodiment:

[0127] During the data acquisition phase, the technical team added 32 high-precision current, voltage, temperature, and vibration sensors to the battery pack management system, drive wheel reduction gearbox, main control MCU, and two sets of LiDAR. All sensor data was pushed to the edge server via MQTT at a frequency of 1Hz. At the same time, the vehicle controller refreshed the vehicle performance indicators such as remaining range, load margin, and positioning accuracy every 10 seconds, and merged them with component-level measurements to generate a six-dimensional component vector and a four-dimensional performance vector.

[0128] During the modeling phase, the project team, based on the unified node and edge specifications of this invention, wrote six types of nodes—battery, motor, drive wheels, LiDAR, controller, and brake—as well as four types of system performance nodes—range, positioning, load, and temperature rise—into TigerGraph. Power supply, mechanical transmission, heat dissipation, and signal links were modeled as 48 directed edges with influence coefficients and fault thresholds. The dynamic twin graph is automatically refreshed every 2 seconds based on incremental data and propagates along the edges in real time to complete the alignment of the entire vehicle's state.

[0129] During the risk reasoning phase, the GNN engine implemented using PyTorchGeometric performs single-round information propagation on the real-time graph. The link margin queue captures the thermal coupling amplification effect of the drive wheels, motor, and battery during the high-load cycle. The edge-level decreasing function and the node-level economic weight operator output the overall vehicle risk score and the set of first-level risk nodes within 80ms. The health vector is then encapsulated into a state frame and pushed to the reinforcement learning environment.

[0130] During the strategy training phase, the SMT line's beat rate and load curves were replicated in a Unity-based digital twin sandbox. An improved PPO algorithm, a three-stage reward function, and risk-suppressing gradient updates were used for training. The agent converged after 260,000 steps, and the exported 1.8MB lightweight weight file was distributed via a private cloud node and accessed in real-time on the shop floor scheduling bus.

[0131] Official operation data shows that the intelligent agent triggered 27 preventative maintenance commands, of which 19 selected reduced-load operation + planned shutdown, and 8 selected immediate repair. Compared with the baseline period, the average AGV downtime decreased from 20 hours per month to 15.6 hours, and the production line availability increased by 6.3%. Graph inference accurately predicted 14 cascading risks of drive wheels, motors, and batteries, with a hit rate of 94.1%. The annual spare parts consumption budget was reduced by 13.7% after optimization. Since all core calculations are completed locally on the private cloud, the processing latency of a single vehicle is controlled within 120ms, meeting the real-time requirements of high-speed material handling scenarios.

[0132] Examples demonstrate the comprehensive applicability of the method of this invention in high-cycle, complexly coupled discrete manufacturing environments for equipment health assessment, risk chain capture, and economically optimal decision-making, laying the model and strategy foundation for subsequent expansion to plant-wide AGV deployment on a larger scale.

[0133] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. An AGV full life cycle intelligent operation and maintenance method based on twin mapping and reinforcement learning, characterized in that, Comprise the following steps: Step 1, real-time acquisition of real-time state data of AGV key components in workshop operation and system performance data output by AGV vehicle-mounted controller, and input of the real-time state data and system performance data into the pre-constructed twin graph to form a dynamic twin graph synchronized with the production line operation; The construction of the twin graph comprises: setting key component nodes and system performance nodes, establishing directed edges with influence coefficients and fault triggering thresholds for energy supply, mechanical transmission, heat dissipation and signal coupling relationship between nodes, and writing all nodes and edges into a graph database to obtain the twin graph; Step 2, input the dynamic twin graph into the graph neural network, propagate once through the graph neural network, record the remaining capacity of the upstream node based on the link residual queue during the propagation process, and determine the risk signal triggered by the node load exceeding the limit, aggregate the risk signal using the edge-level dynamic attenuation function and the node-level economic weight operator to obtain the health vector containing the whole vehicle risk score and component level risk identification, normalize and first difference the health vector to construct the risk increment data, and map the risk component number and the whole vehicle risk score to obtain the economic index, splice into a fixed-length state frame and push to the reinforcement learning environment; Step 3, in the reinforcement learning environment, for the state frame synchronized with the field working condition in the digital twin sandbox, taking immediate repair, load reduction operation, and continuous operation as the action space, training the reinforcement learning agent according to the three-stage reward function containing the production line immediate income, health improvement income and future risk deduction, using the profit-oriented gradient update method with risk suppression factor to optimize the strategy until the discounted income sequence converges to derive the operation and maintenance strategy weight file, and managing the AGV equipment operation and maintenance in the workshop according to the strategy weight file. 2.The AGV full life cycle intelligent operation and maintenance method of claim 1, wherein, In step 1, the key component nodes include battery nodes, motor nodes, laser radar nodes, drive wheel nodes, controller nodes and brake nodes, and the system performance nodes include endurance mileage nodes, positioning accuracy nodes, load capacity nodes and temperature rise margin nodes. 3.The AGV full life cycle intelligent operation and maintenance method of claim 1, wherein, In step 2, the link residual queue is proposed for three types of key paths of energy supply, mechanical transmission and heat dissipation, and the recording of the remaining capacity of the upstream node based on the link residual queue comprises: in the same propagation period, the difference between the current load of the upstream node and the design capacity is written into the corresponding path queue to obtain the remaining capacity of the upstream node.

4. The AGV full life cycle intelligent operation and maintenance method of claim 3, wherein, In step 2, determining the risk signal triggered by the node load exceeding the limit comprises: in the same propagation period, when the difference between the current load of the upstream node and the design capacity is greater than 0, it is determined that the risk signal triggered by the node load exceeding the limit, and the record is immediately marked as a risk signal. 5.The AGV full life cycle intelligent operation and maintenance method of claim 4, characterized in that, In step 2, the edge-level dynamic attenuation function for the risk signal comprises: recalculating the dynamic attenuation weight of each directed edge on the risk signal propagation path, and explicitly introducing the upstream node remaining capacity and fault triggering threshold to construct a decreasing function based on the static influence coefficient; The edge-level dynamic attenuation function is: where ω ij (τ) is a directed edge e ij at risk signal propagation path ij actual transmission intensity at time τ, γ i is the impact coefficient of step 1 solidification; Δr i (τ) is an upstream node v ij remaining amount at time τ; θ ij failure trigger threshold value of failure propagation along e 6.The AGV full life cycle intelligent operation and maintenance method of claim 5, characterized in that, In step 2, the node-level economic weight operator is: where t k is the single repair duration of the component corresponding to node k; v k is the carrying output value created by the component per unit hour; are the in-degree and out-degree of the node in the dynamic twin graph respectively; P k is the set of risk propagation edges pointing to the node; is the cumulative edge-level risk intensity currently borne by the node, e jk is the risk propagation directed edge pointing to node k.

7. The AGV full life cycle intelligent operation and maintenance method according to claim 6, characterized in that, In step 2, the economic index is obtained by mapping the number of risk components and the vehicle risk score, including: The mapping table is used to convert the first-level risk identifier set into a downtime cost estimate, and the vehicle risk score is converted into a line efficiency loss rate, and the economic index is determined according to the downtime cost estimate and the line efficiency loss rate; A set of primary risk indicators is identified into a stoppage cost estimate C pre The piecewise function is: where c d,j represents the single downtime cost of component j, and ζ is the risk component number fold-increase threshold value. According to the stoppage cost estimate C pre and the line efficiency loss rate D los Determine the economic index E idx The piecewise function is: Wherein, α, β are maintenance cost, production line revenue weight, γ is risk over threshold penalty coefficient, R thr is risk score penalty threshold, R veh is the whole vehicle risk score. 8.The AGV full life cycle intelligent operation and maintenance method of claim 1, wherein, In step 3, the three-section reward function includes the immediate income of the production line, the health improvement income, and the deduction of future risks, and the reward signal is designed as a three-section structure of immediate income, health improvement income, and chain risk deduction: where v u is the unit handling output, Q mat is the current cycle effective handling volume, v u Q mat represents the line immediate revenue, ΔR is the difference between the last cycle score and the current vehicle risk score, representing the health improvement magnitude, c l is the downtime cost coefficient, used to convert the health improvement magnitude into economic revenue, δ is the future loss discount rate, E idx is the future downtime loss estimate, γ is the excess risk penalty coefficient. 9.The AGV full life cycle intelligent operation and maintenance method of claim 8, wherein, In step 3, the gradient update method with profit orientation and risk suppression factor includes: The state sequence, action sequence and reward sequence collected by the agent in a batch are written into the replay buffer; First, calculate the discounted cumulative income for each sequence, then read the downtime loss estimate sequence of the same batch, and the net profit trajectory is obtained by subtracting the two; The economic baseline is obtained by averaging all net profit trajectories, and then the economic advantage quantity of each step is calculated by subtracting the economic baseline from the net profit trajectory of each step, and multiplied by the logarithmic probability gradient of the corresponding action to generate the profit-oriented gradient; At the same time, the frequency of the first-level risk node is counted, and the suppression factor is obtained by subtracting the frequency of the first-level risk node from 1, and the gradient is scaled by the suppression factor to form a risk suppression gradient set; The update vector is obtained by summing up the risk suppression gradient set, and the fixed step is applied to the strategy network parameters, and after the update, the high-load scenario is played back in the digital twin sandbox immediately: if the average net profit is not lower than the last update and the cumulative time of the first-level risk is not increased, the parameter is confirmed to take effect, otherwise, the old weight is rolled back and the fixed step is reduced to reiterate. 10.The AGV full life cycle intelligent operation and maintenance method of claim 1, wherein, In step 3, according to the strategy weight file, the AGV in the workshop operation is managed for equipment maintenance, including: the strategy weight file and the action mapping table are packaged into a lightweight running package and uploaded to the private cloud decision node of the workshop, after receiving the real-time state frame, the strategy function is called to generate preventive maintenance instructions including action code, planned downtime and spare parts demand list, and is parallelly issued to AGV controller and operation and maintenance system to complete automatic execution and work order closed loop.

Images